EA Project taking long time to open

Post Reply
richardbyrom
Posts: 4
Joined: 26 May 2011, 09:56
Contact:

I have installed the server based edition of essential and when trying to open the EA model it is taking a long time to open - this is unrelated to network as I'm opening the project on the same server its installed. Can anyone tell me what other areas I need to check to try and reduce the time it takes for the project to open.
User avatar
jonathan.carter
Posts: 1087
Joined: 04 Feb 2009, 15:44

Hi,

I'd like to check that although it is taking a long time to connect, clients can eventually connect to the server and access the repository?

If not, check that the metaproject for the server has been setup correctly. There are tutorials in the Documentation part of the site that cover this.

Other things to consider:
  • Are you using a file-based or database backend for your project that is hosted on the server? If you are using the file-based repository, this demands far more memory and generally provides poor performance as the repository grows. We recommend using a database backend for all multi-user installations
  • Does the platform that is running the server and the client at the same time have sufficient resources to do so? Again, this can be rather memory intensive, so a reasonably sized repository can require a lot of memory. To run both the client and server on the same platform, you should be looking at having 2-4GB of RAM.
  • Watch the system activity while the connection is taking a while to establish. Is the memory or CPU load high? When Protege is working hard, the CPU for Java can go to 100% and you should see the memory allocation going up.
  • Check the console logs on both the client and server for any exceptions. As it starts up the server will be quite verbose about what it is doing. However a successful startup ends with something like 'ready to accept connections'.
  • Check any caching options defined on the client end. The Protege team have done a lot of work over the last couple of years to improve the client/server behaviour, reducing the amount of data that is exchanged to make things more efficient. I've just checked in my Protege v3.4.4 client on OS/X but I can't find any specific caching options but it's worth checking the properties defined in the File->Preferences
This should give you some things to look at.
Additionally, you could try opening other Protege projects on the server. The default Protege installation includes a couple of demo projects, 'newspapers' and 'pizza'. Try connecting to those and seeing how they load. Are they faster or just the same as your Essential repository?

Jonathan
Essential Project Team
richardbyrom
Posts: 4
Joined: 26 May 2011, 09:56
Contact:

Thanks, if I open the demo and EA meta model directly from Protege they are quick to open and the console tells me the time it has taken. However if I access them via the protege client installed on the server using a login the EA projects takes a long time and the console gives the following message - it seems to hang for a while when it was preloading the frame values.

Preloading frame values: DefaultKnowledgeBase(KB_668963)
Preloading frame values: DefaultKnowledgeBase(KB_506241)
WARNING: Annotation/Change project for rmi://GLOESPVM001/BBGHO+EA+Repository not
configured on server (use annotationProject slot) -- ChAOKbManager.getChAOKbFro
mServer()
UI display time = 0 sec
User avatar
jonathan.carter
Posts: 1087
Joined: 04 Feb 2009, 15:44

Sounds like you are running the client and the server on the same host.
Depending on how powerful a machine you have - in particular memory - running both the client and the server on the same machine can make for very large memory requirements. If possible, check the system monitoring of your platform, e.g. Windows Task Manager, to see what the memory usage looks like and whether a lot of paging is going on.

Another thing to try would be to check the performance when using the Protege client on another machine, connecting to the Protege server. Does that also take a long time to start up?

I'd also like to check what your server configuration is set up for in terms of using the Change History capability. Have you got this enabled - via the Changes Tab and Collaborative Protege capability - and is the server metaproject configuration setup correctly in terms of the changes ontology?
Protege uses a separate repository (ontology) to track and manage all the changes and collaboration activity on your repository.

Those last messages suggest that there's an incomplete configuration of the changes (Annotations) ontology and this could be causing Protege to attempt to connect to it, fail and timeout, causing a delay. Most likely, there's an entry in the metaproject configuration that defines an annotations project for your Essential Architecture Manager repository but this 'annotations' project hasn't been set up.
This might be starting to sound a bit complex, but in practice it works well and makes sense. Let me know if you've got any questions about this or it isn't making sense!

Have a look at these things and we'll take it from there.

Jonathan
Essential Project Team
manikvijay
Posts: 20
Joined: 04 Apr 2011, 18:19

Hi,

I am also running into the same issue.

When i open the project from repository within the same server where database is hosted Protege client takes 2-3 minutes to load.

But at the same time if i use a client (desktop) to connect to the repository it takes for ever and not loading at all.

Interestingly if i try to open the Annotation project that is deployed in repository from the client it takes roughly around 8-9 minutes to open it.

So i will check my system usage as you suggested and will post back the results..?
User avatar
jonathan.carter
Posts: 1087
Joined: 04 Feb 2009, 15:44

It's interesting that you mention the Annotations project.

Can I confirm, have you switched on the Collaborative Project functionality and defined the Annotations Project?
If so, make sure to convert the Annotations Project to use the database backend as Protege provides very fine-grain change logging (which can be very useful) and this creates very large numbers of instances, which rapidly uses all the available memory on the server.
The Protege team recognised that the annotations repository gets very large very quickly and note that if you are using a file-based backend for the changes ontology (that's the ChOA stuff in the message) this is managed in-memory and can cause problems.

Assuming that you both have this issue, I would suggest switching off the change history and/or the Collaborative Protege functionality and see whether that makes your project load any quicker.

If you are NOT using the Collaborative Protege and Change history capabilities, make sure that you have NOT defined an Annotations repository for your Essential Architecture Manager project definition in the metaproject. If you've got a reference to an annotations repository that does not exist, that may also cause the server problems.

Hope this helps

Jonathan
Essential Project Team
manikvijay
Posts: 20
Joined: 04 Apr 2011, 18:19

Hi Jonathan,

I unconfigured the Annotation project from our repository project and attended to load (within the server itself). There is no change in loading the project, it takes 2mins to load the project with a warning saying that "Annotation/Change project for rmi://server name/ project name not configured on server -- chAOKbManager.getChAOKFromServer()"

And my CPU is running at 65-80%, and physical memory usage was 60%.

Let me know if i am missing anything here.
User avatar
jonathan.carter
Posts: 1087
Joined: 04 Feb 2009, 15:44

It seems that the system is attempting to make a connection to an Annotations repository - at least it's complaining that it can't find one.

Check that within your Essential Repository project (e.g. essential_baseline_v1.3.pprj) in Protege that you do not have the Track Changes option switched and or that the Collaborative Protege functionality is turned off.

However, after taking 2 minutes (which is unusually long) does the client successfully connect to the server?

First thing to check is that all the collaborative and change-tracking options are disabled. From the error message that you are seeing, it looks like something is trying to connect to an annotations project and failing.
I think if we can eliminate all the change history / collaborative things then we will be able to get to the bottom of why it's taking so long to connect the client and server. I've worked with repositories containing 1000's of instances that connect in just a few seconds.

Jonathan
Essential Project Team
manikvijay
Posts: 20
Joined: 04 Apr 2011, 18:19

Sorry,

where is the option to turn off track changes...?
User avatar
jonathan.carter
Posts: 1087
Joined: 04 Feb 2009, 15:44

No problem.

First, go into Protege and open your repository project (e.g. essential_baseline_v1.3.pprj)

From the menus, select Project->Configure...

This will open a new tabbed dialog. Go to the 'Options' tab and make sure that the 'Track Changes' option is UN-checked.

Have you set up the Collaboration options in this Protege project?
If you, you will have gone through Collaboration->Show Collaboration Panel and set up the 'Annotations Ontology'. If you go in there, there is an 'Enable Change Tracking' option in there that you should UN-check.

However, if you haven't gone through this leave your project as it is and don't set this up.

Once we're sure that the track changes is switched off (Project-> Configure), I'd like to get to the bottom of the warning that you are getting about the chAOKbManager.getChAOKFromServer(). Assuming that the track changes is disabled along with the Collaboration capabilities I have a feeling that this down to how the metaproject is set up.

If it helps, you could ZIP up and send me your metaproject.pprj, metaproject.pins and metaproject.pons files and I'll take a look and make sure that we've got the Annotations project switched off.

Last question from me for now. How big is the repository that you are trying to host on your server? That is, roughly how many instances are in there?
A quick way to find this out is in stand-alone (as well as multi-user) mode to go Project->Metrics on the menus and you'll see a table that includes the total number of instances.

Jonathan
Essential Project Team
manikvijay
Posts: 20
Joined: 04 Apr 2011, 18:19

Thanks for the reply.

I have not added any instances yet. Just using the base essential metamodel now. I renamed the baseline metamodel and hosted it in the SQL server.
You do not have the required permissions to view the files attached to this post.
manikvijay
Posts: 20
Joined: 04 Apr 2011, 18:19

Hi Jonathan,

Sorry for all along. I just got in to your words of saying instances, the baseline model from essential has 25000 instances. And i tried to host the whole project that was pretty heavy that i missed your words before (sorry for that). so now i created a brand new project with only one instance then the client loads like a champ. Thanks for your help.

However, the problem of latency prevails as we grow our metamodel in the future.

I really appreciate your help all along.

Thanks
-Mani
User avatar
jonathan.carter
Posts: 1087
Joined: 04 Feb 2009, 15:44

Hi Mani,

Thanks for getting back to me. 25,000 should be no problem. The Protege team have tested to over 100,000 instances with no issues.

There are some tuning parameters to look at to improve things, though. Firstly, if you are persisting your repository in the PPRJ/PINS/PONT files approach, then the entire repository is loaded into memory, which can take some time if you've got a lot of instances. You'll notice this even more in multi-user mode if the server is running and using a file 'backend'.

Using a database backend really improves things because Protege takes a more efficient approach to loading and caching instances. We recommend that you use a database backend for the server (if you haven't already) in multi-user mode.

In multi-user mode, there are some parameters that you can set on the clients to affect their approach to caching. The later versions (3.4.x) include compression of the 'conversation' between the client and server which I think is activated by default. Worth checking that, too.

Hope this helps

Jonathan
Essential Project Team
User avatar
jonathan.carter
Posts: 1087
Joined: 04 Feb 2009, 15:44

Hi Mani,

I've had a look at the Protege project files that you upload (thanks for that) and this has helped suggest some other things that need looking at.

Firstly, although it might sound like a good idea, I don't think it's necessary to switch on Track Changes or the Collaboration panel for the metaproject. Just to re-iterate, the purpose of this ontology is to manage the configuration of the server and that is all. I would suggest that you do not want to have collaborative, concurrent editing of this ontology, rather that you should be restricting access to it and only make edits to it when the server is stopped.

That is, to make a change:
  • Stop the Protege Server
  • Start a Protege client in stand-alone mode and directly open the metaproject.pprj file (you can do this from the server desktop or if the filesystem is shared, you could do this remotely).
  • Make your changes and save the metaproject
  • Close Protege client
  • Restart the Protege Server
I wonder if this is related to that warning you saw? Was the server still running as you added that user?

By the way, the Protege team have been in recent versions making it possible to perform things like adding users and so on while the server is still running but this is done via the server administration option when you connect to the Protege server, not by directly editing the metaproject. I'm not sure of how 'in production' these capabilities are and in which version of Protege that they work. Worth checking their site.

So, when I opened your metaproject, I was immediately prompted to create the ChAO (changes ontology) for the metaproject. This is because 'Track Changes' is switched on in the Project options. Turn this off and close the Collaboration Panel and this should clear the need for the ChAO - and I now think that it is THIS changes repository that the server was complaining about not being able to find (as there wasn't one for your metaproject).
As I say, I think the best way forward is to turn off track changes and collaboration for the metaproject ontology.

The definition of the Essential AM Project in the metaproject looked fine and I notice that you have removed the Annotations project specification for this project (as we discussed in earlier posts).
However, I did notice that in the definition of the Annotations Essential AM Project that you had specified a .pprj file that was also included in the pack you posted. You had specified to store this ChAO project in the server area (which isn't necessarily a problem) but I thought is was worth pointing out that the changes / annotations repository (ChAO) is specific to each Protege project. That is, Essential AM Project would have it's own one - you do not share the ChAO across multiple repositories as they are created from within the selected Protege project by switching on Track Changes or the Collaborations Panel.

So, in the metaproject, we are telling the server where to find the project files that are already available. It will not create the ChAO for a project, you have to have created that already from within the selected project and then tell the metaproject where that ChAO (file, database etc.) can be found.

I would recommend that rather than manage all the ChAO annotations projects together, e.g. next to the metaproject, you keep the annotations project in the same folder as the Essential repository project. So, if you decide to enable change tracking / collaboration, open your Essential repository, switch these options on and Protege will prompt you to specify where and how to store the ChAO. Once you've done that and Protege has created the annotations project, then you can go back to the metaproject (having stopped the server first) and update the path to the annotations project for the Annotations Essential AM project to point to where you saved the ChAO.

I appreciate that this is a bit fiddly to set up but I think that it all makes sense once you understand how Protege works with this. Hope this post helps and as I say, I think that if we can straighten out your metaproject, you will improve the connection time as I think at the moment, the server is trying to connect to the Annotations project but there is only a PPRJ file (project definition) for that and no content for it, e.g. RDF files etc - as far as I can tell, this has been configured to use the 'changes.rdf' format (rather than a database). So the server is getting stuck unable to successfully open a partially configured project or something.

Jonathan
Essential Project Team
User avatar
jonathan.carter
Posts: 1087
Joined: 04 Feb 2009, 15:44

A quick update on this thread.

We've established that there is an issue with Protege 3.4.6 that is causing the long time (over 1 minute 30 seconds) to initially open the Essential repository when working in multi-user mode.

This is being investigated and in the meantime, we recommend that all multi-user installations use version 3.4.4 of Protege.

Jonathan
Essential Project Team
User avatar
jonathan.carter
Posts: 1087
Joined: 04 Feb 2009, 15:44

Just a quick update on this one.

Recently, the Protege team released version 3.4.8 and we can confirm that this issue has been resolved in this latest (at the time of writing) version.

Jonathan
Essential Project Team
ulfl
Posts: 15
Joined: 01 Feb 2010, 10:47

We recently started to use the Change and Annotation Ontology (ChAO) with our Protege multi-user installation. It worked fine for a while but now we see serious performance degradation. The ChAO is persisted in an Oracle database and the table current contains about 195.000 entries after using it for a few weeks. Currently opening a project with an attached ChAO is impacted the most (up to 70 minutes). The client log file indicates that opening the ChAO for the project takes the longest time. We did some further analysis and found that a server job called "GetSortedTopLevelChangesJob", which is started by the Changes Tab of an Protege client, issues many individual prepared database statements, which take a lot of time to complete. We created additional indexes to mitigate the problem and brought the time for opening a project to about 15 minutes, which is still not acceptable.

Did anyone experience similar performance problems while using the ChAO and has any idea how to improve it? Archiving the ChAO every few weeks is not an option since we would have to search for changes over many archived ChAOs. An archive interval of once a year would be acceptable. How do others handle archiving of ChAO information?
ulfl
Posts: 15
Joined: 01 Feb 2010, 10:47

I did get a reply with a possible workaround on the Protege-Discussion Mailing list:

https://mailman.stanford.edu/pipermail/ ... 06224.html

Best regards,
Ulf
Post Reply