[import Utility 2.12] Publication Issues

[tkinte]

Hi.

After the publication of an import activity from the import utility, I have an error message in the viewer (6.8 and 6.7) :

Code: Select all

 The XML repository snapshot file is empty ...

But the viewer does not throw an error message after the publication of the same updated repository from Protégé.

I see warnings in the ...stderr and catalina log files on the server.

Code: Select all

18-Dec-2019 09:46:13.584 INFOS [main] org.apache.catalina.startup.Catalina.start Server startup in 70570 ms
18-Dec-2019 10:12:04.106 INFOS [http-nio-8180-exec-1] org.zkoss.util.resource.impl.LabelLoaderImpl.loadLabels Loading labels for fr_FR
18-Dec-2019 10:12:04.109 INFOS [http-nio-8180-exec-1] org.zkoss.util.resource.impl.LabelLoaderImpl.loadLabels Loading labels for fr
18-Dec-2019 10:12:04.113 INFOS [http-nio-8180-exec-1] org.zkoss.util.resource.impl.LabelLoaderImpl.load Opening file:/C:/Program%20Files/EA_HOME/Apache%20Software%20Foundation/8109_import_Tomcat9/webapps/essential-import-utility-212/WEB-INF/i3-label.properties
18-Dec-2019 10:17:08.405 AVERTISSEMENT [Thread-6] edu.stanford.smi.protege.plugin.PluginUtilities.findPluginsDirectory Plugins directory not found: C:\Program Files\EA_HOME\Apache Software Foundation\8109_import_Tomcat9\plugins
18-Dec-2019 10:17:26.505 AVERTISSEMENT [Thread-6] edu.stanford.smi.protege.model.PropertyMapUtil.getJavaClassObject Unable to load class: edu.stanford.smi.protegex.widget.graph.GraphPoint
18-Dec-2019 10:17:26.506 AVERTISSEMENT [Thread-6] edu.stanford.smi.protege.model.PropertyMapUtil.getJavaClassObject Unable to load class: edu.stanford.smi.protegex.widget.graph.GraphPoint
18-Dec-2019 10:17:26.513 AVERTISSEMENT [Thread-6] edu.stanford.smi.protege.model.PropertyMapUtil.getJavaClassObject Unable to load class: edu.stanford.smi.protegex.widget.graph.GraphPoint

Code: Select all

18-Dec-2019 09:45:22.069 INFOS [localhost-startStop-1] org.zkoss.zk.ui.sys.ConfigParser.parseConfigXml:159 Loading system default
18-Dec-2019 09:45:22.272 INFOS [localhost-startStop-1] org.zkoss.zk.ui.sys.ConfigParser.parse:275 Parsing file:/C:/Program%20Files/EA_HOME/Apache%20Software%20Foundation/8109_import_Tomcat9/webapps/eaimport611/WEB-INF/zk.xml
18-Dec-2019 09:45:23.241 AVERTISSEMENT [localhost-startStop-1] org.zkoss.zkex.init.WebAppInit.init:29 This is an evaluation copy of ZK EE and will terminate after sixty days from the first date of installation. Should you require an open source license or commercial license for ZK EE please contact us at [email protected] for more information. Alternatively you can download ZK CE which is licensed under the LGPL.
18-Dec-2019 09:45:23.272 INFOS [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive C:\Program Files\EA_HOME\Apache Software Foundation\8109_import_Tomcat9\webapps\eaimport611.war has finished in 20 102 ms

Code: Select all

18-Dec-2019 10:12:04.106 INFOS [http-nio-8180-exec-1] org.zkoss.util.resource.impl.LabelLoaderImpl.loadLabels Loading labels for fr_FR
18-Dec-2019 10:12:04.109 INFOS [http-nio-8180-exec-1] org.zkoss.util.resource.impl.LabelLoaderImpl.loadLabels Loading labels for fr
18-Dec-2019 10:12:04.113 INFOS [http-nio-8180-exec-1] org.zkoss.util.resource.impl.LabelLoaderImpl.load Opening file:/C:/Program%20Files/EA_HOME/Apache%20Software%20Foundation/8109_import_Tomcat9/webapps/essential-import-utility-212/WEB-INF/i3-label.properties
18-Dec-2019 10:17:08.405 AVERTISSEMENT [Thread-6] edu.stanford.smi.protege.plugin.PluginUtilities.findPluginsDirectory Plugins directory not found: C:\Program Files\EA_HOME\Apache Software Foundation\8109_import_Tomcat9\plugins
18-Dec-2019 10:17:26.505 AVERTISSEMENT [Thread-6] edu.stanford.smi.protege.model.PropertyMapUtil.getJavaClassObject Unable to load class: edu.stanford.smi.protegex.widget.graph.GraphPoint
18-Dec-2019 10:17:26.506 AVERTISSEMENT [Thread-6] edu.stanford.smi.protege.model.PropertyMapUtil.getJavaClassObject Unable to load class: edu.stanford.smi.protegex.widget.graph.GraphPoint

Best regards,

Tshitshi

[jonathan.carter]

Hi Tshitshi,

Thanks for your post. To try to understand what’s causing the issue that you’re seeing, I’ll playback what I’ve done to investigate it.

I’ve just tried open source Viewer 6.8 with open source Import Utility 2.1.2.
I set my Import Utility up to point to the relevant Viewer (via the Import Settings -> Target Environments and then editing the selected environment). Note that when you update a ‘LIVE’ target environment, you will need to upload a new Zip of your reference repository.

I ran an import in the Live environment and once it was complete, used the “Publish to Viewer” button. When I checked the Viewer, it was showing my updates. I ran this again for the Test Environment that I’d set up and again the import ran and then published successfully to Viewer, via the “Publish to Test Viewer”.

I saw the same warnings that you have shared (thanks!) - these can be safely ignored in this case - and when I browsed the Viewer, the results of the import were shown.

Looking at the error message that you are seeing in Viewer, it seems that there is an invalid character in the reportXML.xml file after the Import Utility has published. This is likely to be a data issue - and the exception message suggests that this is a character encoding issue.

When you publish to Viewer from within Protege, is it the same dataset?
Is it precisely the same content?
Can you export the result of running the Import as a DUP file and then Import that DUP via the Essential Update tab?

Whilst looking into this I have updated one of our libraries in the Import Utility and tested this with Unicode characters beyond the ASCII set without any issues.

Here is a link to a pre-release version of the Import Utility. From what you’ve described and the tests that I’ve done successfully, I think this should resolve your issue. Download this pre-release version (make to sure to preserve this web-application name - do NOT over-write your current version!) and please let us know whether this resolves the issue.

Pre-release Import Utility 2.1.3.

If this does not resolve your issue, please tell us more about your dataset (if you can share some of your spreadsheet content, that would be great). Also, if the pre-release version does not resolve your issue, I recommend that you publish to Viewer directly from Protege once you complete the import - either via an export as a DUP or by downloading the resulting Protege repository and loading this into Protege.

Regards

Jonathan

[xeonk]

Hi Jonathan,

I've got the same error even after updating to the latest 2.1.3 version.

Here are my error message:

"Oops! The XML repository snapshot file is empty. Please try to re-publish your repository snapshot from the modelling environment.
com.sun.org.apache.xerces.internal.impl.io.MalformedByteSequenceException: Invalid byte 1 of 1-byte UTF-8 sequence."

However if I run through Protege it is working.

Appreciate your advise.

Thanks

[JohnM]

Hi, is there any data you can share so we can replicate this - you can email us if you prefer? The team have tried but can't seem to get the error. The suspicion is that some characters you are using are causing this but they can't work out which.

[tkinte]

Hi John.

Happy New Year!
And Thank you very much to all the EAS team for the support and the improvements during the the past year.

Not sure it is a data issue.

I have got the same error with data from EAS demo viewer and a very basic import script (9 Application Provider).

Please find the Downloaded Repo and the import config (Too large to be attached)

https://drive.google.com/open?id=1lN1NM ... Qm9a1bx7Gt

Best regards.

Tshitshi

[JohnM]

Thanks Tshitshi, Happy New Year to you too.

The team are looking into this now and we'll get back to you.

John

[jonathan.carter]

Thanks for sharing the repository and the Import Utility Configuration for the Import that you're using.

I've applied the EIC to my local open-source Import Utility and then imported the repository that you provided into the configuration.

This gave me the Import Activity, with associated source spreadsheet and the start-point repository for your scenario.

I did find that I had to increase the memory that was available to the local tomcat server that I was using (Tomcat 8) as 2GB was insufficient when loading the repository when I ran the Import (it was fine when I selected the newly uploaded repository as the source repository). However, once I had increased the Java heap and re-ran the Import Utility, the import ran successfully using the Test repository.

When that had completed, I then published directly to the Test Viewer and the publish completed successfully. I then browsed the updated Viewer, confirmed via the timestamp in the footer that it had been successfully updated. I was able to browse any View without any errors.

Next, I ran the same import as a "production" import, uploading the repository that you provided at this point as the source repository. This completed successfully and I then pushed the button to publish to the associated Viewer. This completed successfully and when I browsed the Viewer, the timestamp in the footer was updated correctly and the suite of application provider instances that you supplied were shown in the Application Catalogue.

I have not been able to replicate the issue that you are experiencing.

Could you tell us what version of Tomcat you are running, which operating system that you are running on?
Would it be possible to share a copy of the reportXML.xml file from Viewer that is causing the error message that you are seeing?

Thanks

Jonathan

[tkinte]

Hi Jonathan.

Thank you for the support.

My tomcat config :

Code: Select all

08-Jan-2020 23:11:34.582 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Server version:        Apache Tomcat/9.0.0.M15
08-Jan-2020 23:11:34.588 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Server built:          Dec 5 2016 13:44:25 UTC
08-Jan-2020 23:11:34.589 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Server number:         9.0.0.0
08-Jan-2020 23:11:34.589 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name:               Windows 10
08-Jan-2020 23:11:34.589 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version:            10.0
08-Jan-2020 23:11:34.589 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture:          amd64
08-Jan-2020 23:11:34.589 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home:             C:\Program Files\Java\jre1.8.0_181
08-Jan-2020 23:11:34.589 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version:           1.8.0_181-b13
08-Jan-2020 23:11:34.589 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:            Oracle Corporation
08-Jan-2020 23:11:34.589 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:         C:\Program Files\EA_HOME\Apache Software Foundation\8109_import_Tomcat9
08-Jan-2020 23:11:34.589 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:         C:\Program Files\EA_HOME\Apache Software Foundation\8109_import_Tomcat9
08-Jan-2020 23:11:34.589 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=C:\Program Files\EA_HOME\Apache Software Foundation\8109_import_Tomcat9
08-Jan-2020 23:11:34.592 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=C:\Program Files\EA_HOME\Apache Software Foundation\8109_import_Tomcat9
08-Jan-2020 23:11:34.592 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=C:\Program Files\EA_HOME\Apache Software Foundation\8109_import_Tomcat9\temp
08-Jan-2020 23:11:34.592 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
08-Jan-2020 23:11:34.592 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=C:\Program Files\EA_HOME\Apache Software Foundation\8109_import_Tomcat9\conf\logging.properties
08-Jan-2020 23:11:34.592 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: exit
08-Jan-2020 23:11:34.593 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xms3072m
08-Jan-2020 23:11:34.593 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx4096m
08-Jan-2020 23:11:34.593 INFOS [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xss256k

And The report.xml :


Best regards.

Tshitshi

[jonathan.carter]

Thanks very much for sharing this with us.

I have downloaded the ZIP and removed it from your post to prevent any security issues with the content.

Jonathan

[tkinte]

Hi Jonathan.

Well done.

Best regards.

Tshitshi

[jonathan.carter]

Hi Tshitshi,

Thanks again for sharing the reportXML file.
Using this with my Viewer I can replicate your issue. Comparing this file with the one that I generated from the Import Utility 2.1.2 yesterday, the your file has not been encoded correctly and a large number of symbols are not being encoded as UTF-8 characters in the XML - e.g. copyright symbol, accented characters, etc. - some your content and some of our own pre-loaded content.

To confirm, if you load the repository into Protege and publish to Viewer, everything works fine?
But if you publish directly from the Import Utility, the repository XML is being created incorrectly and you get the error in Viewer?

Some more questions for you:

- Which version of Tomcat are you running for:
(a) Import Utility?
(b) Viewer?

- Did you only start experiencing this issue with Import Utility 2.1.2?

The updated, pre-release version of the Import Utility (2.1.3) should have resolved any encoding issues when publishing from the Import Utility but you have confirmed that it does not.
Did you deploy the 2.1.3 version as a new application alongside the 2.1.2 version?

Which operating system are you running on?

Thanks very much for your help and patience

Jonathan

[tkinte]

Hi Jonathan.

Please find the info inline alongside your questions

Best regards,

Tshitshi


To confirm, if you load the repository into Protege and publish to Viewer, everything works fine?
I confirm

But if you publish directly from the Import Utility, the repository XML is being created incorrectly and you get the error in Viewer?
I confirm

Some more questions for you:

- Which version of Tomcat are you running for:
(a) Import Utility? Apache Tomcat/9.0.0.M15

(b) Viewer? Apache Tomcat/9.0.0.M15

- Did you only start experiencing this issue with Import Utility 2.1.2?
I confirm

The updated, pre-release version of the Import Utility (2.1.3) should have resolved any encoding issues when publishing from the Import Utility but you have confirmed that it does not.
I confirm it does not

Did you deploy the 2.1.3 version as a new application alongside the 2.1.2 version?
I confirm

Which operating system are you running on?

• Apache Tomcat/9.0.0.M15

• Server built: Dec 5 2016 13:44:25 UTC

• Server number: 9.0.0.0

• OS Name: Windows 10

• OS Version: 10.0

• Architecture: amd64

• JVM Version: 1.8.0_181-b13

• JVM Vendor: Oracle Corporation

Thanks very much for your help and patience

[jonathan.carter]

Thanks very much for this.

Which version of the Import Utility were you running before moving to version 2.1.2?

I note that you're using a pre-release version of Tomcat 9. We're testing with a full release (from 2019) of Tomcat 9 shortly. Have you been using Tomcat 9.0.0 for several months/years?

Regards

Jonathan

[tkinte]

Hi Jonathan,
Please find the info inline alongside your questions

Best regards,

Tshitshi

Which version of the Import Utility were you running before moving to version 2.1.2?
The last one released before the 2.1.2

I note that you're using a pre-release version of Tomcat 9. We're testing with a full release (from 2019) of Tomcat 9 shortly. Have you been using Tomcat 9.0.0 for several months/years?
Months
Same issue when tested with apache-tomcat-8.5.50 and apache-tomcat-9.0.30.
Missing test with a different JRE version ... or different machine


Regards

Jonathan

[jonathan.carter]

Thanks very much for these details

Jonathan

[jonathan.carter]

So far, we have been unable to replicate the issue but we are looking into this.

While we're working on this, please use the following approach:
- download the resulting repository from the Import Utility
- load this into Protege
- publish to your selected Viewer from the Essential Viewer tab in Protege

Jonathan

[JohnM]

Quick update, we've finally found the issue, it's to do with how Windows handles some unicode characters. The guys are working on a fix - it's a bit fiddly but we'll come back once it's done.

[tkinte]

Hi John.

Good news! Thanks!

Best regards

Tshitshi

[jonathan.carter]

Hi Tshitshi,

We've released the updated Import Utility and Essential Widgets, which resolve the publishing issue that you identified on Windows.

You can download the latest versions of these components from the Downloads page.

Thanks very much for your help with this

Regards

Jonathan

[tkinte]

Hi Jonathan.

Thank you for the update and the support.

I confirm the Import Utility 2.13 resolve the publishing issue .

Normal Publish from whithin Protégé, after the the installation of the Widgets 6.7.

Best regards

Tshitshi