Cloud Control: OSB targets disappearing after patch

We are currently using Enterprise Manager Cloud Control version 12.1.0.3 to monitor all our database and Fusion Middleware targets. The Oracle Service Bus environments always show one target being in pending state. I can remove this target, but it would reappear after (automatic) refresh of the Weblogic domain.

Investigating this further, reveals that something is wrong in the discovery of the OSB targets.

The target ‘osb_server2_osb’ is visible twice in the navigation tree. Once as target type ‘Oracle Service Bus’ (similar to the soa-infra of the SOA Suite) and once as a ‘Clustered Application Deployment’.

So, the ‘Clustered Application Deployment’ is an illegal target and should not be present. Time to open a Service Request with Oracle Support. We needed to supply a trace on both the agent and the OMS (server) side.

Enabling a trace on the agent can be achieved through Cloud Control.

Changing the log level on the OMS side involves one command.

emctl set property -name log4j.rootCategory -value 'DEBUG, emlogAppender, emtrcAppender' -module logging

After uploading all information to support, I was asked to patch Enterprise Manager to the latest patchlevels. I proceeded and patched the agents, the Oracle Management Server and all deployed plugins (12.1.0.5).

Unfortunately, this patch did not fix my problem. It actually made things worse. Now all targets in the OSB domain (farm) were gone from the navigation tree. Rediscovery did not help. You could see the targets when going to the ‘all targets’ view, but when navigating to any target of the OSB domain, it would show up empty.

There is a workaround to this problem: The illegal target type ‘Clustered Application Deployment’ for osb_server2_osb should be removed.

After this target is removed, the OSB targets appear again. Make sure you don’t (automatically) refresh your Weblogic domain, as that would bring back the offending target.

Oracle support is aware of this issue and working on a fix (bug 18042645).

Update
Oracle has released a fix for this issue through FMW Plug-in Bundle Patch 12.1.0.5.3 and FMW Plug-in 12.1.0.6.0: See MOS note  1625453.1

References

Cloud Control: Automatic refresh of Weblogic domain members
Bug 18042645 : The folders in the target navigation panel are empty