[Essential V6.5] Problem with JSON string Escaping in roadmap enabled views

[jmk]

Hi,
the roadmap enabled views are great!

However these views are unusable in French as there are many problems with JSON string escaping in the released code (v6.5.3).

So, I would like to know the intent of the following two XSL functions (defined in the file common/core_js_function.xsl) :

eas:removeQuotesFromString : this function is sometimes applied to strings whose content is already JSON-escaped, such as here

Code: Select all

'description': '<xsl:value-of select="eas:removeQuotesFromString($planDesc)"/>',

eas:getSafeJSString : this function replace '.' by '_' in the argument string, it is sometimes applied to JSON-record field such as here

Code: Select all

id: "<xsl:value-of select="eas:getSafeJSString(current()/name)"/>",

To be true, IMHO, these calls are harmful/useless ...


Regards,

Jean-Marie.

[jonathan.carter]

Hi Jean-Marie,

Thanks for your post.

We will take a look at these functions.

Can I confirm: Is it strings with apostrophe's in - the ' character - that are being broken by these operations?

Thanks

Jonathan

[tkinte]

Related topic

https://www.enterprise-architecture.org ... m.php?f=21

[jmk]

Hi Jonathan,

Can I confirm: Is it strings with apostrophe's in - the ' character - that are being broken by these operations?

eas:removeQuotesFromString

1) Strings with apostrophe are indeed broken by a call to eas:removeQuotesFromString if the strings are already escaped. n this case, the viewer is broken and nothing is displayed in the view.

2) The meaning of Strings with apostrophe is modified when eas:removeQuotesFromString is used. In this case something is displayed by the viewer, but ths 'something' is not what could be expected ...


eas:getSafeJSString

This function doesn't break strings, it's intended usage is just not clear (who would want to replace dots by underscores in strings only (it seems) because they are JSON strings ?)

I asked the question when i was trying to underrstand how the IT asset report works ... As a quick workaround I defined a RenderJSXXXX template for each RenderXXX defined in common/core_utilities and modified template calls in relevant xsl files ...

Regards,

J.-M.

[jonathan.carter]

Hi Jean-Marie,

Thanks again for raising this issue. We've fixed the eas:removeQuotesFromString function (removed it and replaced it with a built-in XSL function).

We're going to leave the eas:getSafeJSString() function in as it is used in a number of places to dynamically generate some Javascript. As you said, it is is not breaking anything and we couldn't find a quick built-in replacement that worked everywhere.

I've attached a ZIP file of updated components that remove the eas:removeQuotesFromString() function and replace all calls to it with a built-in XSL function. We've tested this with combinations of strings that contain ' and " as well as accented and other characters.

We felt that, as you are using open-source, it was fastest and simplest to provide you with the updated components which will, of course, be included in the next release.

There are 4 XSL files that need to be updated:

• <TOMCAT>/essential_viewer/common/core_js_functions.xsl
• <TOMCAT>/essential_viewer/common/core_roadmap_functions.xsl
• <TOMCAT>/essential_viewer/enterprise/core_el_strategic_implication_dashboard.xsl
• <TOMCAT>/essential_viewer/technology/core_tl_tech_product_list_by_vendor_simple.xsl

The ZIP file contains these 4 files. Please copy each to the locations described above.

Jonathan

[tkinte]

Hi Jonathan.

The fix does not solve the issue for me. I still have the error "Uncaught SyntaxError: Unexpected identifier".

The JSON snippet :

Code: Select all

		'id': 'essential_normalize_unicode_issue_v65_Class3',
		'name': 'Accijnsinrichtingen-Etablissements d'accises_Prod',
		'link': '<a href = \"?XML=reportXML.xml&PMA=essential_normalize_unicode_issue_v65_Class3&cl=en-gb\" class = \" context-menu-appProviderGenMenu\" id =\"essential_normalize_unicode_issue_v65_Class3Link\" >Accijnsinrichtingen-Etablissements daccises_Prod</a>',
		'description': 'Inputapplicatie ter ondersteuning van de functionaliteit van de Algemene website GandI, meer bepaald de mogelijkheid om de geldige vergunningen accijnsinrichtingen te consulteren.<br/><br/>Een inputapplicatie die, door medewerkers van de dienst Gegevensbeheer, gebruikt wordt om de afgeleverde vergunningen accijnsinrichtingen te registreren. Van zodra de toepassing KIS-SIC de mogelijkheid biedt om dergelijke vergunningen af te leveren, wordt de functionaliteit van deze toepassing redundant. Deze applicatie zal hierdoor op korte termijn buiten gebruik worden gesteld.<br/><br/>Planned disposed retired.',
		'menu': 'context-menu-appProviderGenMenu',

The apostrophe in the instance name is causing the trouble, for Application Catalogue view;

But the core_int_all_instances_by_class.xsl renders correctly the same instance.

Thank you for the support.

Best regards.

Tshitshi

[jonathan.carter]

Thanks very much for pointing this out.

We're working on a fix for this now and will post back the updated components

Jonathan

[jonathan.carter]

Hi Tshitshi,

Thanks again for bringing this issue to our attention.

Please find a pack of updates for your Viewer in the attached ZIP file.

There are 19 XSL files in this pack and so I've organised each file into folders in this ZIP:

• application/api
• business/api
• common
• common/api
• enterprise
• technology
• technology/api

Copy the XSL files into the corresponding folder within your Essential Viewer and these will resolve the rendering of quotes and accented characters.

Jonathan

[tkinte]

Hi Jonathan.

Thank you very much for the support.

The fix works partially. It is OK for the provided instance data.

Unfortunately, it is not comprehensive enough. Other normalisation needs remain uncovered or not correctly covered.

To help test fixes and profile the issue, I provide you with some "strange" instances data.

Best regards

Tshitshi

dataset_4_unicode_json_normalize.zip

[jonathan.carter]

Thanks for these, we'll look into the examples that you've sent.

Jonathan

[jonathan.carter]

Hi Tshitshi,

Thanks again for sending through the set of examples. We were able to replicate the issues that you were seeing and have resolved them, such that all these examples now render correctly in Viewer.

Only very small changes to the following 2 files were required:

• common/core_utilities.xsl
• application/core_al_app_provider_list_table.xsl

I've included the updated versions in the attached ZIP (which also includes the files that were updated in the last pack I sent - just to keep everything together).

Copy the above 2 files into the common and application folders respectively in your Viewer.

Jonathan

[tkinte]

HI Jonathan.

Thank you very much.

The normalization seems ok.

AppPro Catalog By Name : OK
AppPro Catalog By AppFam : OK

BusProc Catalog : OK

Still failing to render : App Pro Catalog As Table

Code: Select all

Failed to compile stylesheet. 1 error detected.

Fatal Error: No template exists named RenderModalReportContent; SystemID: file:/C:/Program%20Files/Apache%20Software%20Foundation/Tomcat%209.0_Tomcat9_viewer/webapps/minfinea/application/core_al_app_provider_list_table.xsl; Line#: 75; Column#: 68

In addition, I have a Proxy Error 502 when trying the fix behind a Reverse Proxy.


Best regards.


Tshitshi

[neil.walsh]

Hi Tshitshi,

The code Jonathan shared here has an extra line which will only work with a forthcoming release. I've removed the problem lines and reattached zip here.

Let us know how you get one with these

Cheers

Neil

[tkinte]

Hi Neil,

Thank you.

Worked fine.

The code Jonathan shared will be part of Viewer 6.6 ? As the eup is already visible on the download area.

Best regards

Tshitshi

[neil.walsh]

Hi,

Meta model 6.6 is available however 6.6 Viewer is not yet ready.

Cheers,

Neil

[tkinte]

Thanks!

[tkinte]

Hi .

the Viewer 6.5.5 still has this the issue.

The previously provided fix seems to work for this version 6.5.5 too.

Unfortunately, the use of that fix with the viewer 6.5.5 causes another issue on the portal links (Home | Enterprise Architecture | View Library) :

Code: Select all

Oops!  Essential Viewer encountered an error with the request View Template XSL stylesheet

Failed to compile stylesheet. 1 error detected.

Fatal Error: No template exists named RenderEditorLinkText; SystemID: file:/C:/Program%20Files/EA_HOME/Apache%20Software%20Foundation/8080_Viewer_Tomcat9/webapps/essential_viewer-6.5.5/common/portal_template.xsl; Line#: 458; Column#: 56

Best regards.

Tshitshi

[neil.walsh]

Hi Tshitshi,

We've release version 6.7 today which has lots of improvements around the handling of JSON in views as well as many other features and updates. I can't guarantee it will fix all your issues but it should provide an improved experience.

https://www.enterprise-architecture.org/os_download.php

It would be great if you could let us know if you're still having any JSON or language related problems with this release as we would be keen to get them fixed as soon as we are able.

Thanks

Neil

[tkinte]

Hi Neil,

Sorry for the delay.

Congratulations on this new version. It brings many significant improvements and new features.

JSON Issue : Fixed.

Language and History issues : Fixed when running without a reverse proxy.

I Still have Language and History issues when running the viewer with a reverse proxy.

Best regards.

Tshitshi