Issues with data-access at bundle initialization time

[frankjoppe]

Hi,

We have an application, and upgraded from smx4.5.3 to smx6.1.0. We also
upgraded from dbcp to dbcp2 and pool2. One thing I noticed after this
upgrade, was that bundle start-up and shutdown was very, very slow. So I
looked around with Google and found that Spring-DM is deprecated and will be
replaced with Blueprint. I also replaced Spring-DM to Blueprint for all the
bundles, and now they are much faster in start-up and shutdown.

However Blueprint has some differences. For example, We've made Camel
routebuilders and in the configre() method, we get some configuration from
the Database. Or we have beans with a set-ter, which we give the database
connection, and we also do some init-from-the-DB in this setter method.

We get exceptions. The MySql Drive class cannot be found. The bean using the
DB connection (ie the configure() method in our Camel route) gets some
"proxy", while the other side of the proxy has not been initialized yet. So
packages fail when SMX starts. And if you manually restart the bundle, it
works.

My analysis is that the DB call is done too early and that not everything
has been initialized yet.

How to solve this?

The DB connection is initialized in a blueprint xml, and added to jdni.

What I already tried is:
* The bundle blueprint has a "<reference " to the DB connection, and I used
a reference-listener - sometimes works, sometimes not;
* Added a dependency in "import-packages" in the pom.xml, on pool2, under
the maven-bundle-plugin section, coincidently found this, I've seen good
results but not sure if it is 100% waterproof;
* Everywhere when possible, used lazy-inits, this works 100% but cannot be
applied for the Camel Routebuilder configure() method (I don't know when
this is called);

Things I did not try are:
* Put configuration in local files - for this release unfeasible, for future
an option;
* Revert to Spring-DM, I still consider this as an option, but I have seen
its behavior in my machine only, and I don't know if it has the same
problems as Blueprint, still an option.. but Blueprint has my preference.

Please Help



--
View this message in context: http://servicemix.396122.n5.nabble.com/Issues-with-data-access-at-bundle-initialization-time-tp5723674.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.

[Morgan Hautman]

Hi Frank,

If I understand it correctly, you have multiple bundles where some are
camel routes and 1 of them is a proxy bundle, right?

If so, you can make a Karaf feature and use start-levels.
So your proxy-bundle that is required can startup prior to the others, so
it will be initialized when the other startup.

But in all cases don't revert to Spring-DM.

Regards,
Morgan

Post by frankjoppe
Hi,
We have an application, and upgraded from smx4.5.3 to smx6.1.0. We also
upgraded from dbcp to dbcp2 and pool2. One thing I noticed after this
upgrade, was that bundle start-up and shutdown was very, very slow. So I
looked around with Google and found that Spring-DM is deprecated and will be
replaced with Blueprint. I also replaced Spring-DM to Blueprint for all the
bundles, and now they are much faster in start-up and shutdown.
However Blueprint has some differences. For example, We've made Camel
routebuilders and in the configre() method, we get some configuration from
the Database. Or we have beans with a set-ter, which we give the database
connection, and we also do some init-from-the-DB in this setter method.
We get exceptions. The MySql Drive class cannot be found. The bean using the
DB connection (ie the configure() method in our Camel route) gets some
"proxy", while the other side of the proxy has not been initialized yet. So
packages fail when SMX starts. And if you manually restart the bundle, it
works.
My analysis is that the DB call is done too early and that not everything
has been initialized yet.
How to solve this?
The DB connection is initialized in a blueprint xml, and added to jdni.
* The bundle blueprint has a "<reference " to the DB connection, and I used
a reference-listener - sometimes works, sometimes not;
* Added a dependency in "import-packages" in the pom.xml, on pool2, under
the maven-bundle-plugin section, coincidently found this, I've seen good
results but not sure if it is 100% waterproof;
* Everywhere when possible, used lazy-inits, this works 100% but cannot be
applied for the Camel Routebuilder configure() method (I don't know when
this is called);
* Put configuration in local files - for this release unfeasible, for future
an option;
* Revert to Spring-DM, I still consider this as an option, but I have seen
its behavior in my machine only, and I don't know if it has the same
problems as Blueprint, still an option.. but Blueprint has my preference.
Please Help
--
http://servicemix.396122.n5.nabble.com/Issues-with-data-access-at-bundle-initialization-time-tp5723674.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.

[frankjoppe]

Hi,

Thanks for responding. No, we don't have a bundle which acts like a proxy. The proxy - as far as I can guess - is a Blueprint thing (see http://aries.apache.org/modules/blueprint.html under section "Reference Dynamism").

In the deploy folder, we have a file called jndi.xml, and this is a fragment of its contents:

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0"
<cm:property-placeholder id="database.properties" persistent-id="our.database.connection.properties"/>


<bean id="datasourceMain" class="org.apache.commons.dbcp2.managed.BasicManagedDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver"/>
<property name="url" value="${connection-string}"/>
<property name="username" value="${connection-username}"/>
<property name="password" value="${connection-password}"/>
<property name="maxTotal" value="20"/>
<property name="maxIdle" value="10"/>
<property name="maxWaitMillis" value="5000"/>
<property name="minEvictableIdleTimeMillis" value="2880000"/>
<property name="testWhileIdle" value="true"/>
<property name="validationQuery" value="select 1"/>
<property name="transactionManager" ref="jtaTransactionManager"/>
</bean>
<service id="serviceMade" interface="javax.sql.DataSource" ref="datasourceOurApp">
<service-properties>
<entry key="osgi.jndi.service.name" value="MainDB"/>
</service-properties>
</service>


Then we have a maven compiled bundle, with blueprint.xml initialization in the OSGI-INF.blueprint folder (resources), with this fragment:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://camel.apache.org/schema/blueprint
http://camel.apache.org/schema/blueprint/camel-blueprint-2.16.1.xsd
">
<bean id="ourlogicBean" class="oprg.unimportant.SomeClass">
<property name="dataSource" ref="dataSourceRef"/>
</bean>
<reference id="dataSourceRef" interface="javax.sql.DataSource" filter="(osgi.jndi.service.name=MainDB)"/>
</blueprint>

(please forgive spelling mistakes, I'm cutting/pasting this on the fly, my issue is not in syntax)


I have witnessed two cases when NoClassDefFound is thrown:

1. When the method "setDataSource" in SomeClass does a call to the database;

2. When the method "setDataSource" in SomeClass only registers the value in a backing field, but SomeClass is a (Camel) RouteBuilder and the DB is accessed from the configure() method;

I have set a breakpoint in case 1 above and the value received is non-null, so I guess this is the blueprint proxy mentioned above.

Hope this explains it a bit better.

Regards,
Frank




Van: Morgan Hautman [via ServiceMix] [mailto:ml-node+***@n5.nabble.com]
Verzonden: woensdag 11 mei 2016 9:48
Aan: Joppe, Frank
Onderwerp: Re: Issues with data-access at bundle initialization time

Hi Frank,

If I understand it correctly, you have multiple bundles where some are
camel routes and 1 of them is a proxy bundle, right?

If so, you can make a Karaf feature and use start-levels.
So your proxy-bundle that is required can startup prior to the others, so
it will be initialized when the other startup.

But in all cases don't revert to Spring-DM.

Regards,
Morgan

Post by frankjoppe
Hi,
We have an application, and upgraded from smx4.5.3 to smx6.1.0. We also
upgraded from dbcp to dbcp2 and pool2. One thing I noticed after this
upgrade, was that bundle start-up and shutdown was very, very slow. So I
looked around with Google and found that Spring-DM is deprecated and will be
replaced with Blueprint. I also replaced Spring-DM to Blueprint for all the
bundles, and now they are much faster in start-up and shutdown.
However Blueprint has some differences. For example, We've made Camel
routebuilders and in the configre() method, we get some configuration from
the Database. Or we have beans with a set-ter, which we give the database
connection, and we also do some init-from-the-DB in this setter method.
We get exceptions. The MySql Drive class cannot be found. The bean using the
DB connection (ie the configure() method in our Camel route) gets some
"proxy", while the other side of the proxy has not been initialized yet. So
packages fail when SMX starts. And if you manually restart the bundle, it
works.
My analysis is that the DB call is done too early and that not everything
has been initialized yet.
How to solve this?
The DB connection is initialized in a blueprint xml, and added to jdni.
* The bundle blueprint has a "<reference " to the DB connection, and I used
a reference-listener - sometimes works, sometimes not;
* Added a dependency in "import-packages" in the pom.xml, on pool2, under
the maven-bundle-plugin section, coincidently found this, I've seen good
results but not sure if it is 100% waterproof;
* Everywhere when possible, used lazy-inits, this works 100% but cannot be
applied for the Camel Routebuilder configure() method (I don't know when
this is called);
* Put configuration in local files - for this release unfeasible, for future
an option;
* Revert to Spring-DM, I still consider this as an option, but I have seen
its behavior in my machine only, and I don't know if it has the same
problems as Blueprint, still an option.. but Blueprint has my preference.
Please Help
--
http://servicemix.396122.n5.nabble.com/Issues-with-data-access-at-bundle-initialization-time-tp5723674.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.

________________________________
If you reply to this email, your message will be added to the discussion below:
http://servicemix.396122.n5.nabble.com/Issues-with-data-access-at-bundle-initialization-time-tp5723674p5723675.html
To unsubscribe from Issues with data-access at bundle initialization time, click here<http://servicemix.396122.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5723674&code=ZnJhbmsuam9wcGVAa3BuLmNvbXw1NzIzNjc0fC0xOTg5ODA3ODQ0>.
NAML<http://servicemix.396122.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>




--
View this message in context: http://servicemix.396122.n5.nabble.com/Issues-with-data-access-at-bundle-initialization-time-tp5723674p5723676.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.

[frankjoppe]

Hi,

As an addition to this response: I did try with bundle priority before and it did not work.

I think it works like this:
One bundle is initialized in one thread, receiving a proxy to the MySql driver, while the backing MySql driver is initialized in another thread, and they are not ready at the same moment and in the right order. So if the bundle init is ready before the DB-connection (and pooling?) init, we are calling a MySql proxy class without backing. I think that causes the NoClassDefFound exception. I could not think of another explanation. Also, the behavior is different on different systems, even having the same OS and Java version. Also packages which fail are different on every system.

Bundle prio could not change this.

Regards,
Frank


Van: Joppe, Frank
Verzonden: woensdag 11 mei 2016 11:04
Aan: 'Morgan Hautman [via ServiceMix]'
Onderwerp: RE: Issues with data-access at bundle initialization time

Hi,

Thanks for responding. No, we don't have a bundle which acts like a proxy. The proxy - as far as I can guess - is a Blueprint thing (see http://aries.apache.org/modules/blueprint.html under section "Reference Dynamism").

In the deploy folder, we have a file called jndi.xml, and this is a fragment of its contents:

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0"
<cm:property-placeholder id="database.properties" persistent-id="our.database.connection.properties"/>


<bean id="datasourceMain" class="org.apache.commons.dbcp2.managed.BasicManagedDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver"/>
<property name="url" value="${connection-string}"/>
<property name="username" value="${connection-username}"/>
<property name="password" value="${connection-password}"/>
<property name="maxTotal" value="20"/>
<property name="maxIdle" value="10"/>
<property name="maxWaitMillis" value="5000"/>
<property name="minEvictableIdleTimeMillis" value="2880000"/>
<property name="testWhileIdle" value="true"/>
<property name="validationQuery" value="select 1"/>
<property name="transactionManager" ref="jtaTransactionManager"/>
</bean>
<service id="serviceMade" interface="javax.sql.DataSource" ref="datasourceOurApp">
<service-properties>
<entry key="osgi.jndi.service.name" value="MainDB"/>
</service-properties>
</service>


Then we have a maven compiled bundle, with blueprint.xml initialization in the OSGI-INF.blueprint folder (resources), with this fragment:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://camel.apache.org/schema/blueprint
http://camel.apache.org/schema/blueprint/camel-blueprint-2.16.1.xsd
">
<bean id="ourlogicBean" class="oprg.unimportant.SomeClass">
<property name="dataSource" ref="dataSourceRef"/>
</bean>
<reference id="dataSourceRef" interface="javax.sql.DataSource" filter="(osgi.jndi.service.name=MainDB)"/>
</blueprint>

(please forgive spelling mistakes, I'm cutting/pasting this on the fly, my issue is not in syntax)


I have witnessed two cases when NoClassDefFound is thrown:

1. When the method "setDataSource" in SomeClass does a call to the database;

2. When the method "setDataSource" in SomeClass only registers the value in a backing field, but SomeClass is a (Camel) RouteBuilder and the DB is accessed from the configure() method;

I have set a breakpoint in case 1 above and the value received is non-null, so I guess this is the blueprint proxy mentioned above.

Hope this explains it a bit better.

Regards,
Frank




Van: Morgan Hautman [via ServiceMix] [mailto:ml-node+***@n5.nabble.com]
Verzonden: woensdag 11 mei 2016 9:48
Aan: Joppe, Frank
Onderwerp: Re: Issues with data-access at bundle initialization time

Hi Frank,

If I understand it correctly, you have multiple bundles where some are
camel routes and 1 of them is a proxy bundle, right?

If so, you can make a Karaf feature and use start-levels.
So your proxy-bundle that is required can startup prior to the others, so
it will be initialized when the other startup.

But in all cases don't revert to Spring-DM.

Regards,
Morgan

Post by frankjoppe
Hi,
We have an application, and upgraded from smx4.5.3 to smx6.1.0. We also
upgraded from dbcp to dbcp2 and pool2. One thing I noticed after this
upgrade, was that bundle start-up and shutdown was very, very slow. So I
looked around with Google and found that Spring-DM is deprecated and will be
replaced with Blueprint. I also replaced Spring-DM to Blueprint for all the
bundles, and now they are much faster in start-up and shutdown.
However Blueprint has some differences. For example, We've made Camel
routebuilders and in the configre() method, we get some configuration from
the Database. Or we have beans with a set-ter, which we give the database
connection, and we also do some init-from-the-DB in this setter method.
We get exceptions. The MySql Drive class cannot be found. The bean using the
DB connection (ie the configure() method in our Camel route) gets some
"proxy", while the other side of the proxy has not been initialized yet. So
packages fail when SMX starts. And if you manually restart the bundle, it
works.
My analysis is that the DB call is done too early and that not everything
has been initialized yet.
How to solve this?
The DB connection is initialized in a blueprint xml, and added to jdni.
* The bundle blueprint has a "<reference " to the DB connection, and I used
a reference-listener - sometimes works, sometimes not;
* Added a dependency in "import-packages" in the pom.xml, on pool2, under
the maven-bundle-plugin section, coincidently found this, I've seen good
results but not sure if it is 100% waterproof;
* Everywhere when possible, used lazy-inits, this works 100% but cannot be
applied for the Camel Routebuilder configure() method (I don't know when
this is called);
* Put configuration in local files - for this release unfeasible, for future
an option;
* Revert to Spring-DM, I still consider this as an option, but I have seen
its behavior in my machine only, and I don't know if it has the same
problems as Blueprint, still an option.. but Blueprint has my preference.
Please Help
--
http://servicemix.396122.n5.nabble.com/Issues-with-data-access-at-bundle-initialization-time-tp5723674.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.

________________________________
If you reply to this email, your message will be added to the discussion below:
http://servicemix.396122.n5.nabble.com/Issues-with-data-access-at-bundle-initialization-time-tp5723674p5723675.html
To unsubscribe from Issues with data-access at bundle initialization time, click here<http://servicemix.396122.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5723674&code=ZnJhbmsuam9wcGVAa3BuLmNvbXw1NzIzNjc0fC0xOTg5ODA3ODQ0>.
NAML<http://servicemix.396122.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>




--
View this message in context: http://servicemix.396122.n5.nabble.com/Issues-with-data-access-at-bundle-initialization-time-tp5723674p5723679.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.