RTC EE deployment: where distributed and mainframe developers can work in the same place !

February 19, 2014

Last month, along with my colleague Tim Wilson, I met with a couple of customers in China. I wrote a previous blog post based on this business trip that recalled the importance of using the right terminology for the tool adoption.

Customer under the impression that distributed and mainframe teams required separate CLM / RTC servers (i.e. JTS)

During a discussion with another customer, it came up that the person in charge of SCM administration (for distributed)  was under the impression that distributed and mainframe teams required separate CLM / RTC servers (i.e. JTS).

Since the tool itself does not impose such separation, I made the point this would rather be an organizational decision rather than a technical one. I felt like this was pretty new to the customer.

MainframeDistributedTogether

To clarify things, I initiated a blackboard session to highlight the different options wrt. the deployment topologies, their advantages as well as their implications for the future. Some core messages were:

  • The RTC server(s) can run on a various environments (including Windows, Linux, AIX, Unix, zLinux, z/OS, IBMi, etc.).
  • The fact that RTC EE is used for developing for z/OS does NOT imply that the RTC server shall run on z/OS. This was already stressed in other resources from IBM colleagues,
  • You have the options of using a single JTS or separate JTSes. In both cases, you can associate multiple CCM instances to your JTS(es). Choices should be guided either based on performance considerations or, again, on organizational considerations.
  • When considering options for using multiple JTSes:
    • It’s very important to note that, once separated, they could NOT be merged later.
    • Two separate servers require doubled effort for administration, upgrade, etc.
    • The reporting tools should be adapted accordingly (e.g. Insight/RPE instead of RRDI/RRDG)

At the end, the customer could have a better understanding of the CLM/RTC EE deployment options for his organization.

For the little story, in this customer situation, SCM administrators for both distributed and mainframe developments were supposed to meet with each others after our visit to discuss the best deployment option. Yet another good effect of ‘Enterprise Modernization’  brought by the adoption of the RTC EE tooling !

Note: in this article, I insisted on some aspects of RTC/CLM deployment. For a complete picture, the Deployment wiki is definitely a good read.


RTC / RTC EE: adopt terminology first !

January 24, 2014

Last week,  I was on a business trip to China with a colleague of mine. We met multiple local IBM teams and visited a couple of financial customers.

During our customer-facing sessions, questions raised by the customer were technical in essence which is… fair enough as we’re part of the RETT (Rational Emerging Technologies Team). But some questions revealed:

  • some misconception about the product (RTC / RTC EE)
  • some reluctance of parts of the customer organization for adopting a newer, yet different but more modern tool with safer workflows.

As an illustration, it appeared that the technical lead in charge of the RTC EE adoption at one of these customers still used the terms of checkin/checkout for RTC SCM when discussing with us.

Despite:

  • RTC does NOT have a concept of checkout (although RTC does support resource locking – currently at the stream-level)
  • Customer organization had started migrating to RTC EE almost 2 years before…

I see multiple drawbacks in this:

  • Sticking to the terminology from a previous tooling in technical discussions about the capabilities of a newer tool is confusing. It basically assumes features that are not present in the product or, at least, not as is… As such this mismatch is a brake for adoption of the new features.
  • Actually, I was even more concerned that this speech was coming from someone in charge of the adoption of our tooling. Assuming that most of developers in the customer organization would share the same terms…

As a result, I decided to clarify and made the point that in RTC, you load projects and components, you checkin change sets, and deliver them to a stream. This change in emphasis made it simpler to discuss business scenarios to be implemented in RTC EE (SCM here) at this customer’s.

Terminology-first

A disconnect in terminology is more important than it seems:

  • It introduces a gap compared to the proper concepts, which are basic prerequisites for elaborating the right workflows in RTC to implement a business scenario.
  • It’s a good indicator that trainings and pilot phases missed at least some targets. Basically the “infusion” of the up-to-date concepts that match the tools used by developers on a day-to-day basis.

I came up with the following ideas:

  • Answering customer initial questions is only ONE piece of the solution.
  • Other aspects (not raised by customers) need to be addressed with the same attention:
    • the return over experience in the training area,
    • the return over experience in the pilot projects,
    • the management of the change (from various point of views: organizational, managerial, etc. ),
    • the adoption in general.

Taking the whole combination into consideration is key for success.

In the contrary case, an ultimate bad effect would be that SCM management simply do not trust their developers wrt. their usage of the new tooling. Possibly triggering… a second bad effect : having the SCM management to try enforcing too much process in the tool to substitute the lack of training of their developer. Not the right route anyway…..

As a recap, here are the simple things that should be kept in mind when adopting RTC / RTC EE:

  • Even if adoption is accomplished in a stepwise manner, make sure everyone uses the RTC terminology asap in your organization.
  • If you’re in a team focusing on adoption of RTC, make sure you replace questions/feedback/concerns of your developer in the correct terminology.

Direct access to Redbooks/redpapers on mainframe and z/OS: improve your learning curve!

November 17, 2013

When digging further (or simply jumping) into Mainframe and z/OS world, one faces new terminology, concepts and tooling. Literature is pretty vast in the area and one could spend some time figuring out the right resource to consider…

As far as terminology is concerned, if you just need to understand a term, the “Glossary of z/OS terms and abbreviations” is certainly indicated. For a mainframe concept, you can check this InfoCenter entry  and for a z/OS concept, this other Info Center entry

If you need a broader understanding (e.g. how the pieces articulate with each others, etc.) then the Redbooks/Redpapers targeting Mainframe and z/OS are good starting points.

In my personal experience during the last couple of months, I found the following content very useful. In the following tables,  you’ll be one click away from the right material (in PDF format). While I’ve never taken time to read all of this content (at least not yet), I’ve referred to it in an “on demand” way, with a pre-established topic in mind for which I was interested by getting a clearer idea on.

Adding that “a picture is worth a thousand words“, so they say, I personally found the summary slides and architectural diagrams (generally located at the beginning of sections) particularly handy and enlightening. Enjoy !

[Redbooks] “ABCs of z/OS Systems Programming”

Vol. 1 Intro to z/OS and Storage concepts, TSO/E, ISPF, JCL
Vol. 2 z/OS Implementation and Maintenance 
Vol. 3 Intro to DFSMS and Storage Management
Vol. 4 Communication Server, TCP/IP, and VTAM
Vol. 5 Base and Parallel Sysplex, Logger, GRS, Operations, ARM 
Vol. 6 RACF, PKI, LDAP, Crypto, Kerberos and Firewall
Vol. 7 InfoPrint Server, LE, and SMP/E 
Vol. 8 z/OS Problem Diagnosis
Vol. 9 z/OS, UNIX System Services 
Vol. 10 Intro to Z, LPAR, and HCD 
Vol. 11 Capacity planning, performance management, RMF, and SMF
Vol. 12 WLM
Vol. 13 JES3 

Other Redbooks:

Batch Modernization on z/OS (July 2012)
Introduction to the New Mainframe: z/OS Basics (March 2011)
System z Mean Time to Recovery Best Practices  (Sept 2010)
System z End-to-End Extended Distance Guide (Nov 2013)

RedPapers:

Rethink Your Mainframe Applications: Reasons and Approaches for Extension, Transformation, and Growth (May 2013)

RTCz / RTC EE evolutions version-by-version

October 3, 2013

Following the citation “One does not really know a science until one knows its history“, I’ve decided to provide such information for my area of interest: RTC Enterprise Extensions.

The story started in 2009 and I felt like providing such a recap could help both customers (and potentially business partners/colleagues from the field) when questioning a particular features “introduced at some point of time…. but when actually ?

It shall be noted that the following information borrows a lot to a previous blog post from Robin Yehle Bobbitt. Robin covered versions 2.0.0.1 to 3.0.1. I’m extending to versions starting from 4.0 up to 4.0.4 versions.

RTC EE

The following picture is somewhat interactive: you could click any circle in it, consult a specific version and then click back on your browser, etc. Hope you enjoy it !

RTC z evolution line

CLICK on ANY VERSION NUMBER !

Follow this link to discover the new feature in the version 3.0 Follow this link to discover the new feature in the version 3.0.1 Follow this link to discover the new feature in the version 3.0.1 Follow this link to discover the new feature in the version 4.0 Follow this link to discover the new feature in the version 4.0.1 Follow this link to discover the new feature in the version 4.0.3 Follow this link to discover the new feature in the version 4.0.4

Version 2.0.0.1 (Q4 2009)

  • z/OS file agent : made it possible to extract files from the SCM to PDS/Es.
  • The mass import tool allowed you to import existing source from data sets into the SCM.
  • Rational Build Agent allowed you to create build definitions that call out to existing host build facilities, or submit JCL.
  • New build type, Antz build, provided full capabilities to build any z/OS related artifacts (via Apache Ant extensions for the z/OS environment).
  • Support for RDz remote projects.
  • Collaborative debugging with RDz and the IBM Debug Tool.
  • Jazz Gateway for System z (now called REST Gateway) to simplify integration with existing host based SCMs.  Allows simple REXX programs or any other exit to programmatically extract information, such as work item status, from the Jazz repository.

(more details) / Back to the “evolution path”

Version 3.0 (Q4 2010)

  • RTCz : not a a separate product anymore. Instead, a “Developer for IBM Enterprise Platforms” license was required for full access to the z and Power capabilities.
  • New “Team Area > Enterprise Extensions” node.
  • ISPF client for “green screen” access to common SCM and build functions.
  • Dependency build capability (for only building programs as necessary, i.e. based on change).
  • Component promotion to move source and build outputs together through the development hierarchy (e.g. Dev->Test->QA->Prod).
  • Shiplist-based packaging and deployment to gather your build outputs and deploy them to another runtime location.

(more details) / Back to the “evolution path”

Version 3.0.1 (Q2 2011)

  • Work-item based promotion, packaging and deployment.
  • Preview capabilities for dependency build and promotion.
  • HFS file system support in the ISPF client.

(more details) / Back to the “evolution path”

Version 4.0 (Q2 2012)

  • Sparse loading in ISPF client.
  • Enhanced view, compare and conflict resolution in ISPF client.
  • IBM i: dependency builds and work-item / component-based promotion and packaging.
  • Dependency build report.
  • More features around packages.
  • Personal deployment.

(more details 1) / Back to the “evolution path”

(more details 2) / Back to the “evolution path”

Version 4.0.1 (Q4 2012)

  • Build management in ISPF client.
  • Builds and source control: patterns to control the naming of translator outputs.

(more details) / Back to the “evolution path”

(more details 2) / Back to the “evolution path”

Version 4.0.3 (Q2 2013)

  • Work-item dissociation in the ISPF client.
  • View/compare file content in the ISPF client.
  • Build subsets configuration and settings.
  • Dependency build automatically rebuilds main programs (when one or more statically linked subprograms are updated).
  • Dependency build tuning and improved performance.

(more details) / Back to the “evolution path”

Version 4.0.4

  • ISPF client: pending changes view.
  • Dependency build and promotion: usability and simulation.

(more details) / Back to the “evolution path”

References

  • Good summary of enterprise features can be found here on jazz.net.
  • Citation reference “One does not really know a science ….” (and background information) can be found here

Credit / Acknowledgment

Thanks for Robin Y. Bobbitt for letting me reuse some information contained in her previous blog post.


I’m part of a new team: the RETT !

September 12, 2013

If you follow my LinkedIn page you might have noticed that I have changed for a new team at Rational.

Actually, I have joined the Rational Emerging Technology Team. Its manager is Daniel Toczala (a.k.a. Tox). While the team focuses on multiple topics (see related blog post here), with a couple of GREAT fellows I’ll personally focus on the RTC EE (Rational Team Concert Enterprise Extension, basically the RTC extension for z/OS development) area .

This is a very exciting challenge as I’ll bring my knowledge of Rational Collaborative Lifecycle Management (CLM) – and Rational Team Concert (RTC) in particular – to our mainframe customers and internal resources.

Don’t be surprised if you see in my next posts some interesting news about RTC EE and other practical tricks, etc.


CLM,RTC,RQM,RRC/RDNG: recommendations and “educated guesses” for limitations

September 5, 2013

As part of the Jumpstart team, I help our customers in their CLM adoption and deployments. Customers raise questions on the sizing of their CLM environment, the topologies to adopt, etc. The questions I hear most often are:

  • How many users (total or concurrent) can my CLM environment support ?
  • RTC rollout at our company is close to reach a second milestone (additional teams will use the tool). What planning (HW/SW) should we have wrt. these modifications?
  • We have this huge number of CLM (RTC/RQM/RRC-RDNG) artefacts. Will my CLM environment still handle this without any performance degradation as we continue adding artefacts into our repository ?
  • What are the intrinsic CLM product limitations and – if one is concerning me – what approach should I adopt to keep on working smoothly with my CLM ?
  • etc.

First of all, depending on the CLM version you’re running, central places to check are the CLM 2011 Sizing Guide and the CLM 2012 Sizing Guide which include an “Artifact Sizing Guidelines” section summarizing “the recommendations on artifact sizing that will ensure optimal performance of the repository when the data sizes increase significantly“.

Foreword to the reader:

  • This post follows the “cheat sheet/how to” format I’ve used in earlier posts for CLM Reporting or OSLC-related topics. As a consequence, if you’re already familiar with this post (and know exactly what you’re looking for), you may want to navigate directly to the tables: JTS tableRTC tableRQM tableRRC/RDNG table.
  • If you’re interested by a similar content for Enterprise Modernization products (i.e. RTC EE, RDz, RD&T, RAA, etc.)  by IBM Rational, check this dedicated blog post.

Now back to the core of this post:

Tables

Tables are provided. OK. But what do we call a Limit and an Alert zone ?

  • Quantified data: the (maximum) number of….
  • Limit: a hard limit of the product. Meaning that you cannot go beyond this value.
  • Alert zone: based on experiences with customers, internal tests and development teams, it’s around these values that we start seeing performance issues. If you’re approaching these values, we’d suggest you monitor your system closely to detect any performance degradation before it becomes critical. WARNING: while provided figures are educated guesses and practical rules of thumbs, you could still find that your environment functions perfectly beyond these limits  (e.g. if your environment is particularly fine-tuned). In a similar way,  some intense CLM usage could show that these recommended values are too optimistic…

As a consequence, it’s important to understand this post is NOT an attempt for replacing existing resources (see the References section) that provide extensive views on CLM performance and tuning topics. We encourage administrators and project managers to read them as they both include finer-grain information and insist on the key aspect of not loosing the “bigger picture”.

What’s the use of the following tables then ?

Answer: they’re here to HELP YOU quickly figure out if you’ve reached some known CLM limitation or if you’re getting close to a threshold  (again: on the basis of a typical/average environment) requiring due monitoring of your environment.  To this regard, these tables are COMPLEMENTARY with existing resources and concentrate information ALREADY available but yet disseminated on multiple medias/sites/forum posts/etc.

What if… you can’t find what you’re looking for in the following tables ?

Answer: in such case, there MAY not be soft/hard limit on it. You should check the References section at the bottom of this post and check for the latest information (esp. from the CLM sizing guides).

Ξ

JTS
Quantified data Limit Alert zone Reference(s)
Concurrent user sessions  – 400-2000+ Purple Book
Jazz user id length  – 250 bytes Jazz.net forum

Ξ

RTC 
Quantified data Limit Alert zone Reference(s)
Concurrent user sessions  –  300-500+ (per CCM instance) Purple Book
   Example: “CLM Sizing Strategy” (v4.0.6 – April 2014)  –   – 100-600 concurrent users. See report/environment details.
   Example: “Performance Report” (v5.0 – June 2014) 1200 concurrent users. See report/environment details.
Planning – Work-items
   WIs in a plan (<= v2.0) 2048 Jazz.net article
   WIs in a plan (>=v2.0.0.1)  –  250-500+ (impacts plan display time + questions ability from user to grasp several 100s of WIs in one plan) Jazz.net forumJazz.net forumPurple Book, RTC 4.0.3 Plan performance improvement
   WIs in a project area/repository  – Jazz.net forum, Jazz.net article
   WI attachment size  50 MB  If increasing this value or systematically using large attachments: be aware of the possible impact on DB growth and CLM performance in general. See how to change this value in TechNote, Jazz.net forum
   WI “Estimate” attribute  1 year  – Jazz.net forum. A  presentation-enforced limit.
   WI custom attribute length > Small String (*) 250 bytes Jazz.net article (RTC v4.0)
   WI custom attribute length > Medium String (*) 1000 bytes Jazz.net article (RTC v4.0)
   WI custom attribute length > Large String (*) 32768 bytes Jazz.net article (RTC v4.0), Enhancement 160469
   WI custom attribute length > Medium HTML (*) 1000 bytes Jazz.net article (RTC v4.0)
   WI custom attribute length > Large HTML (*) 32768 bytes Jazz.net article (RTC v4.0)
Queries
   Query results
 1000 results Jazz.net forum. Note: this default value could be increased but be aware of the possible negative impact on usability / server performance.
Planning – Timelines
   Timelines  2048  (see recommended approach in the forum post’s answer) Jazz.net forum
SCM
   Files/folders in a single component (CLM 2011)  50K (split into multiple components if required) Jazz.net forumJazz.net article
   Files/folders in a single component (CLM 2012, RTC 5.0)  100K (split into multiple components if required) Jazz.net forumJazz.net article, Jazz.net forum,
   Suspended change-sets by individual user  300 (for not slowing down operations) Jazz.net article
   Components in workspaces and streams  500 (as tested by IBM) Jazz.net article , Task 176441 (in progress)
Build
  Build definitions associated to a build engine ( < v4.0.3) 2048 TechNote
OSLC
   oslc_cm.pageSize parameter (when querying work-items) 100 Jazz.net RFEJazz.net forum

(*): text-based

Ξ

RQM
Quantified data Limit Recommendation Reference(s)
Concurrent user sessions  –  100-150+ (per QM instance) Purple Book
   Example: “CLM Sizing Strategy” (v4.0.6 – April 2014)  – 350-500 concurrent users. See report/environment details.
   Example: “Performance Report” (v5.0 – June 2014) 1000 concurrent users. See report/environment details.
TER (Test Execution Record) name length  250  – Jazz.net forum
TCERs bulk generated from test plan wizard  500 Work-around article, RQM defect, WAS maxParamPerRequest
TCERs bulk changed/removed at once tbd  tbd Jazz.net forum, Jazz.net forum, WAS maxParamPerRequest
Records in a datapool / test data 2000  – Jazz.net enhancement
Character limit: Description field of a Lab Resource 250 Jazz.net enhancement
Number of categories defined on an artifact type 50 RQM defect, RQM defect
Feed entries per page ( < 4.0.4)  512  – Jazz.net forum, RQM defect
“Large Record Count” (SQL query result set generated by OOTB BIRT reports) ( >= 4.0.5)  –  10K TechNote, Jazz.net defect
Attachment size using UI Jazz.net enhancement
Attachment size using CLI ( Command-Line Interface) ( >= 4.0)  50 Mo Jazz.net article (for how to change this default value, see the Comments section). Note: if increased, be aware of the possible negative impact on usability / server performance.
TCERs runnable off-line and at once (>=4.0) 50 4.0 InfoCenter

Ξ

RRC-DNG/RDNG
Quantified data Limit Recommendation Reference(s)
Concurrent user sessions (< v4.0.1)  200+ Purple Book
 (>= v4.0.1)  400+ Purple Book
   Example: “CLM Sizing Strategy” (v4.0.6 – April 2014) 300-400 concurrent users. See report/environment details.
   Example:  “Performance Report” (v5.0 – June 2014) 400 concurrent users. See report/environment details.
Coexistence with DM (Design Manager) on the same box (in v.4.x and v5.0.x)  Incompatible  – Jazz.net forum (related to the converter component)
Instances of RM application per JTS (<= v4.0.6) 1 4.0.3 InfoCenter, Jazz.net article, Plan Item
RM Projects per RM application / JTS  200+ Jazz.net article
Number of undos in edit mode  20 TechNote
Number of displayable links (>= v4.0.1)
  • 60 (IE7)
  • 100 (other browsers)
Jazz.net forum
Number of artifacts selectable in the Artifact view 50 Enhancement 71080, Jazz.net forum
Using ReqIF
   Imports to DNG from DOORS (>= v4.0.1)
  • 5000 modules
  • 200K objects (total)
Jazz.net article, Jazz.net forum
   max depth supported for import 3 Jazz.net forum

References:

  1. CLM 2012 Sizing Report

  2. CLM 2011 Sizing Guide
  3. Rational Team Concert (RTC) 2.0 sizing guide
  4. Rational Team Concert 4.x sizing report for z/OS

  5. The Deployment wiki
  6. Jazz Performance: A Guide to Better Performance” by D. Toczala (Feb 2013). A.k.a the “Purple Book
  7. Sizing and tuning guide for RDNG (Rational DOORS Next Generation) 5.0

Acknowledment/Credit: thanks to the authors of the cited documents above and more generally to the Jazz community who collaboratively provides accurate information through library articles, forums questions & answer, etc..


How to best use RRDI data dictionaries for CLM Reporting

August 28, 2013

In the previous post Landing page for CLM Reporting with RRDI or Insight, I recapped on the CLM reporting uses cases options:

  • Check/Use the Out-of-the-box reports
  • Author your own reports

Links on both OOTB reports and RRDI CLM applications dictionaries for multiple versions were provided and chosen concise presentation  received positive feedback.

A feedback I recently heard about was that our customers could sometimes miss the information on how to best use these data dictionaries.

This is actually a NON gap since the CLM Information Center provides such information HERE.

ReportingDataDictionnaryFrom4.xInfoCenter

I recommend you read this topic so that you get a clear understanding of how to use RRDI data dictionaries.

Don’t miss this useful information !

Acknowledgment: thanks to Amanda Brijpaul, from Rational User Technologies team, for pointing me this important resource.


Jazz/CLM ETLs performances: a view on release-to-release comparisons

August 1, 2013

Recently, I’ve had a deeper look at ETLs performances. This topic is sensible to our customer as full ETL load could last up to several days !

Based on work-items tracking activity and interactions with our development, support team and performance teams, I came up with a personal summary. I found it interesting to be shared with Jazz/CLM Administrators to get up to speed on performance of such ETLs.

Firstly, let’s start with the existing and customer-oriented information:

ETLs performance comparison between CLM versions 3.0.1.2 and 4.0.

Available in article “Rational solution for CLM 4.0 “Extract, Transform, and Load” Performance Report“.

It’s considering the “D1” topology (note: the article is little bit misleading as the provided URL link points directly… to the “E1” / Enterprise topology. I have notified the authors already).

A simplified summary (which should not prevent you from reading the full outcome of the ETL comparison) follows:

  • RTC ETLs: stable performance
  • RQM ETLs: 20% performance degradation (expected by dev team)
  • RM DM ETLs: significant improvement
  • Star ETL: no major issue

Secondly, let’s continue with the information provided for advanced readers

ETLs performance comparison between CLM versions 4.0.2 and 4.0.3

Available in [Plan Item 248546] “Ensure no ETL performance regressions are found when comparing CLM 2012 Mod 3 ETLs to 4.0.2“.

It’s again considering the “D1” topology.

Status (as for Aug. 1st, 2013) is “Done”. Navigating to the latest comments of the Discussion tab will show you available Excel spreadsheets (Java ETLs for RRDI, DM ETLs for Insight)  .

Finally, mentioning the plans for improving subsequent ETLs performance comparisons:

ETLs performance comparisons for forthcoming 4.x versions

Performance team have plans (be aware that plans are subject to change) to:

  • publish some automated ETL tests for later minor version of CLM 4.x
  • improve the analysis of performance data from the ETLs.

The related WIs are listed below:

Also citing some material of interest (but to a lesser extent from a customer perspective as these resources are part of the Jazz development wiki) as well:

Note: be aware the previous resources are subject to the following statement: “Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly“.

Finally, mentioning some related information (captured in the Deployment wiki)

…. and recalling a basic statement

If you’re concerned that the ETLs performances may have degraded after a particular CLM upgrade, you would need to keep in mind the amount of RTC WIs, RQM test cases, RRC artefacts, etc. has certainly increased since you last run a full ETLs load. This remark for avoiding comparing apples and oranges…


%d bloggers like this: