Success Stories

 
 

The following is a small selection of customer success stories where SELCOPY was used to accomplish a specific task.

Global Data Processing

Global Data Processing

Objective:

Generate a new text data file containing latest name/address information for all customer entries.

Description:

File1 is an ASCII text data file containing original customer information. Each record in the file contains a unique reference number (URN) and corresponding customer name/address details. (Approx 18 million records in all.)
e.g. 


  1906872547   Ms Jo Bloggs,   3 New Road,   ......

File2 is an ASCII text file with each record containing old and new name/address details for customers referenced in File1. Each record includes two URN, one each for the old and new customer name/address details. Records in File2 are sorted in ascending order of old URN.

Notes:

  • The URN in File1 may match an old URN reference in File2.
  • The corresponding new URN in File2 may also match an old URN in another File2 record.
    (e.g. where a customer has changed address more than once.)

e.g. 


  190687  Ms Jo Bloggs,  3 New Road, ....  652868  Ms Jo Bloggs,  16 Old Street, ...
  652868  Ms Jo Bloggs, 16 Old Street, ..  891427  Mrs Jo Thomas, 42 Castle View, ..
 
Solution:

Since records in File2 are sorted by a unique identifier (the old data URN), SELCOPY keyed read syntax is used to input File2 records directly using the old data URN field as the data key.

  1. Read sequentially a record from File1 to obtain the URN key.
  2. Use the URN key to perform a keyed (direct) read of the appropriate record in File2.
  3. If non-zero new URN exists in the second part of the File2 record, then use it to perform another keyed read of File2.
  4. Repeat this until new URN is zero, at which point we have the customer's current details.
e.g. 


  READ   "c:\data\src_addr.txt"      INTO 1001  * URN at Position 1001.
  ...
  ==LOOP==
    READ "c:\data\upd_addr.txt" KEY=10 AT 1001  KEYPOS=1 KEYLEN=10
                                                * Keyed Read using URN.

    IF POS NEW_URN  <>  0
      THEN  MOVE     10 AT NEW_URN     TO 1001  * New URN Field.
      THEN  GOTO LOOP       * Go back and repeat the keyed Read.
 
Benefits:

SELCOPY's ability to perform direct (i.e. non-sequential) input of a plain ASCII text file of this size reduces processing significantly when compared to using alternative solutions.

This particular job completed in under an hour having processed over 18 million records. Alternative methods using sequential input failed to complete and were cancelled after executing for over 8 hours.

Macro 4 Plc

Macro 4 Plc

Objective:

Convert AWSTAPE format files to real tape in a UNIX/Linux environment.

Description:

Macro 4 had a requirement to produce AWSTAPE format product tapes at certain regional offices. These offices have access to the UK based mainframe systems but do not have the physical tape drives attached locally. Installing the hardware at these locations with channel extenders was an extremely costly option.

Although 3rd party solutions were available to produce S/390 tapes on non mainframe platforms, Macro 4 Internal Systems Group wrote their own application running under Linux to create the product tapes. SELCOPY was an integral part of this application as it could perform the necessary data manipulation and write the files out to tape in the required format.

Macro 4 and Compute (Bridgend) enjoy a long running relationship in the mainframe arena. The Macro 4 project team knew that the power and flexibility of SELCOPY made it an ideal solution for tape creation on the Linux platform. They received first class support from Compute (Bridgend) enabling the application to be successfully tested and the project to move on to the next phase.

Solution:

A UNIX/Linux SELCOPY job was created to read the AWSTAPE file, interpret field information and write file data and appropriate tape marks to a mounted (non-rewinding) tape device.

Using SELCOPY's flexible data input capabilities, discrete amounts of data could be read from the file based on data lengths extracted from header blocks embedded within the data itself.

The length of an AWSTAPE data block is contained within the preceding and following header blocks in Little-Endian format. Header blocks also contain bit flags that indicate a tape mark or the beginning/end of a data block.

A version of the completed SELCOPY job, which faithfully converts data contained in any AWSTAPE file, is published in the table below. Note that the format of a header block is mapped using SELCOPY EQUATE statements at the beginning of the job.


TEXT Format (Opens in new browser) File Size Last Update
AWS2TAPE_CTL - ASCII TEXT 5,371 bytes 2007/04/12 17:32

Since output is to a non-rewinding tape, the SELCOPY CLOSE operation can be used to close the output file in order to write a tape mark and so allow subsequent WRITE operations to write data after the tape mark.

This solution also resulted in a new feature for SELCOPY; the FLUSH operation immediately writes all data in SELCOPY's output buffer to the output device.

Benefits:
  • Eliminates dependency upon S/390 & zSeries emulation hardware and software for AWSTAPE->real tape conversion.
  • The ability to email a virtual tape reduces delivery time and international carrier charges for tape distribution.
  • The SELCOPY job is portable across all supported UNIX platforms with little or no change. Therefore, regional offices are not restricted to operating with the same type of UNIX operating system.
Customer Comment:

"The partnership with Compute (Bridgend) has really added value to our operation. The production and introduction of SELCOPY has saved us considerable sums which can now be invested in other areas to help our business continue to grow."
Martin Solly, Enterprise Systems Group Leader

Statistics Finland

Statistics Finland

Objective:

Migrate from Mainframe to UNIX/AIX without the burden of outsourcing legacy files.

Description:

Statistics Finland, the official statistics producer of Finland, recently decomissioned their mainframe.

Their MVS system had over 150 000 sequential files that contained binary arithmetic fields, which prevented a straightforward move to their UNIX environment (with standard character conversion).

Most of the files had not been used since the 1990's. However, the owners of the data did not dare to delete the files, fearing they would be needed at some point in the future.

A mass conversion would have been prohibitively expensive and wasteful, since Statistics Finland could not anticipate which files would be needed in the future. The only other alternative would have been to outsource all the legacy files to a mainframe service shop, not a cheap solution either.

Solution:

Features of SELCOPY for UNIX enable users to process data sets that have been imported from a mainframe environment. Specifically, these are the ability to process the following:

  • RECFM=V/VB format data. (i.e. Records have BDW and/or RDW prefix).
  • EBCDIC/ASCII data conversion.
  • Big Endian (s/390 & z/Architecture) Binary data.
  • Packed Decimal data.
  • IBM Hex and IEEE Binary format Floating Point fields (New Feature).

Statistics Finland found that SELCOPY suited their specific purpose.

All legacy files were transferred from the mainframe platform to UNIX disk space ("binary FTP style"). It was then verified that SELCOPY for UNIX allows users to easily retrieve, process and manipulate data in these files, and so enable access to the legacy data should it became necessary to do so in the future.

Statistics Finland are very satisfied with this lightweight cost solution, and feel confident that their legacy data will be available for as long as they choose to preserve it!

Benefits:

  • Mainframe files can be kept in-house even after the move from mainframe to UNIX/Windows production environment.
  • Files can be retrieved for processing if (and only if) they are needed.
  • Avoids a costly and cumbersome mass conversion of tens of thousands of (mostly obsolete) mainframe files.

Customer Comment:

"Our specific download environment is IBM AIX. We found out that AIX/Selcopy did not do conversions from mainframe floating point fields. However, CBL promptly filled this gap, and after the required tests, StatFin became a Selcopy user. A benefit for Finnish taxpayers, and a legacy preserved for later use!"
Kauko Hämäläinen, IT Development Manager

Finance

Major, Multi-National Financial Services Company

Objective:

Delete specific segments from an IMS/DL1 database (on a z/OS mainframe system) and allow for a program restart should an unexpected or abnormal end occur.

Description:

Arpit Asthana, a senior consultant working on behalf of a major, multi-national financial services company, regularly performs trouble shooting tasks against IMS/DL1 databases.

On one such occasion, he required a batch process to delete a large number of segments of the same segment type that had been written to an IMS database. In particular, to comply with company standards, the process needed to perform regular check points and support a restart if necessary.

The database segments to be deleted were to be identified using unique key field values, provided as record entries in a sequential (QSAM) data set.

Following consultation between Arpit and the support staff at CBL, a SELCOPY batch procedure was developed that not only performed the required operation, but used a framework of SELCOPY sub-routine modules that could easily be utilised in further SELCOPY IMS/DL1 procedures.

SELCOPY IMS/DL1 Processing Overview:

SELCOPY syntax includes operations that perform IMS/DL1 calls for database management. Specifically, these are segment retrieval calls (GN, GNP, GU) and their hold equivalents (GHN,GHNP,GHU), segment insert (ISRT), replace (REPL) and delete (DLET). In order to retrieve a segment directly by key for subsequent deletion, the GHU (Get Hold Unique) call must be performed.

In order to support checkpoint calls, the PSB specified as a parameter to DFSRRC00 and containing the PCB used by SELCOPY to process the required database, must have been generated using PSBGEN parameter CMPAT=YES. This ensures that the PSB is always treated as if there were an I/O PCB, a pre-requisite for CHKP and XRST calls. Furthermore, to perform Get and Delete operations on the particular database segment type, the PCB selected to process the database must be sensitive to that segment type and have a minimum processing option (PROCOPT) specification of GD.

SELCOPY also supports the CHKP operation which performs the IMS/DL1 system service checkpoint (CHKP) call for a basic checkpoint only. However, in order to restart the SELCOPY program, an extended restart (XRST) call must be performed which requires a checkpoint id generated by an extended (symbolic) CHKP.

A symbolic CHKP call has the additional benefit of being able to save areas of storage that are subsequently restored on execution of XRST with a checkpoint id. In this way, the contents of the SELCOPY user work area buffer may be reinstated following a restart. Therefore, values that record the current location within an input or output object can be maintained in the work area and later used following a restart to re-establish the status quo as at the time of the checkpoint.

Since SELCOPY does not have built-in support for symbolic CHKP and XSRT, they must be invoked via a CALL operation to the IMS/DL1 ASMTDLI assembler program interface with an appropriate list of parameter values. ASMTDLI is a member of the SDFSRESL load library.

Solution:

The IMS DB batch SELCOPY job was written to delete segments referenced by key field values supplied via a sequential input data set (INDD).

Note that, following each direct read of a database segment (GHU), a symbolic CHKP is taken.

If restarting the job following an abnormal termination, specify the checkpoint id in the DLIBATCH procedure parameter list (e.g. CKPTID=SELC0001). Note that this checkpoint id should be that of the last checkpoint taken before the job terminated. See the last DFS0540I system message for the required checkpoint id.

The SELCOPY job step performs the following processing steps:

  1. Execute an Extended Restart operation (XRST) once only.
  2. If no CKPTID is specified then we have a normal start. (Note that an XRST must be performed even on a normal start in order to perform symbolic CHKP operations.) Jump to step 5.
  3. If a CKPTID is specified then the SELCOPY work area will be restored to its status when the specified check point occurred. We must now restore the position of the sequential input file at the record read before the checkpoint was taken.
  4. Because the input record number (INCOUNT) value for INDD was saved in the (now restored) work area before the check point was taken, we can use this value to read forward to the same record number before proceeding with the GHU/DLET processing. Read records from the sequential input data set (INDD) until the INCOUNT value matches the saved input record number. Jump to step 6.
  5. Read a key value from the sequential input data set (INDD). If end-of-file, then the job ends normally.
  6. Use this key value on a GHU operation to read a segment from the input IMS database. If the GHU returns a bad status code, then the status code is reported and the job ends abnormally.
  7. If the GHU is successful, save the current sequential input (INDD) record number in the work area and perform a symbolic checkpoint operation (CHKP). Note that the current input record number for INDD is maintained in an internal 4-byte binary field called INCOUNT. The checkpoint ids generated are of the format "SELCnnnn" starting at SELC0001. Each time the CHKP is performed, nnnn will be incremented by 1.
  8. Delete (DLET) the current input segment. If the DLET returns a bad status code, then the status code is reported and the job ends abnormally.
  9. Jump back to step 5.

SELCOPY SYSIN input is comprised of a number of concatenated library members and in-stream (DD *) data sets, each containing the SELCOPY control statements. The JCL batch job and SYSIN input members are published in the table below.


File Name File Size Last Update
SSJCL 5,308 bytes2016/05/12 12:06
SSCHKEQU1,940 bytes2016/05/12 12:06
SSIN1EQU 593 bytes2016/05/12 12:06
SSGHUEQU 407 bytes2016/05/12 12:06
SSGHU 1,967 bytes2016/05/12 12:06
SSIN1 3,204 bytes2016/05/12 12:06
SSCHK 2,797 bytes2016/05/12 12:06
SSDEL 1,195 bytes2016/05/12 12:06

Benefits:

SELCOPY's IMS/DL1 operations may be used in conjunction with its other data set I/O and data manipulation features to manage IMS DB data.

Use of a library of SYSIN input "plug-in" members which contain either SELCOPY sub-routines or SELCOPY EQUate statements, provides a framework for future SELCOPY IMS/DL1 job development. This enables a programmer to produce new batch jobs to process IMS/DL1 data in very little time, especially when compared to writing a COBOL or PL1 program.

Customer Comment:

"It has been an outstanding support which I received from Nick/CBL who went out of the way and helped me with the understanding of chkp-xrst logic for SELCOPY-DLI code which is a new learning experience. Great Support!!!"
Arpit Asthana, Senior Consultant