Jazz/CLM ETLs performances: a view on release-to-release comparisons

Recently, I’ve had a deeper look at ETLs performances. This topic is sensible to our customer as full ETL load could last up to several days !

Based on work-items tracking activity and interactions with our development, support team and performance teams, I came up with a personal summary. I found it interesting to be shared with Jazz/CLM Administrators to get up to speed on performance of such ETLs.

Firstly, let’s start with the existing and customer-oriented information:

ETLs performance comparison between CLM versions 3.0.1.2 and 4.0.

Available in article “Rational solution for CLM 4.0 “Extract, Transform, and Load” Performance Report“.

It’s considering the “D1” topology (note: the article is little bit misleading as the provided URL link points directly… to the “E1” / Enterprise topology. I have notified the authors already).

A simplified summary (which should not prevent you from reading the full outcome of the ETL comparison) follows:

  • RTC ETLs: stable performance
  • RQM ETLs: 20% performance degradation (expected by dev team)
  • RM DM ETLs: significant improvement
  • Star ETL: no major issue

Secondly, let’s continue with the information provided for advanced readers

ETLs performance comparison between CLM versions 4.0.2 and 4.0.3

Available in [Plan Item 248546] “Ensure no ETL performance regressions are found when comparing CLM 2012 Mod 3 ETLs to 4.0.2“.

It’s again considering the “D1” topology.

Status (as for Aug. 1st, 2013) is “Done”. Navigating to the latest comments of the Discussion tab will show you available Excel spreadsheets (Java ETLs for RRDI, DM ETLs for Insight)  .

Finally, mentioning the plans for improving subsequent ETLs performance comparisons:

ETLs performance comparisons for forthcoming 4.x versions

Performance team have plans (be aware that plans are subject to change) to:

  • publish some automated ETL tests for later minor version of CLM 4.x
  • improve the analysis of performance data from the ETLs.

The related WIs are listed below:

Also citing some material of interest (but to a lesser extent from a customer perspective as these resources are part of the Jazz development wiki) as well:

Note: be aware the previous resources are subject to the following statement: “Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly“.

Finally, mentioning some related information (captured in the Deployment wiki)

…. and recalling a basic statement

If you’re concerned that the ETLs performances may have degraded after a particular CLM upgrade, you would need to keep in mind the amount of RTC WIs, RQM test cases, RRC artefacts, etc. has certainly increased since you last run a full ETLs load. This remark for avoiding comparing apples and oranges…

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: