Installing the Model9 agent
Obtain your license key from Model9 by opening a “License" request in the Model9 service portal: https://model9.atlassian.net/servicedesk/customer/portals
The output of the z/OS command “D M=CPU” is required for obtaining a new license.
Verify that the z/OS Java is version 8 64-bit SR6 FP25 or above. Using an older Java version would result in an agent startup failure with
ZM91005E message code.Note: z/OS Java version 11 is not supported
Verify the Java version and change the Java installation directory using the following commands:
cd /usr/lpp/java/J8.0_64/bin
./java -version
java version "1.8.0_281"
Java(TM) SE Runtime Environment (build 8.0.6.25 - pmz6480sr6fp25-20210115_01(SR6 FP25))
IBM J9 VM (build 2.9, JRE 1.8.0 z/OS s390x-64-Bit Compressed References 20201218_462060 (JIT enabled, AOT enabled)
OpenJ9 - 4c03b71
OMR - 86a8e1a
IBM - 8c30c56)
JCL - 20210108_01 based on Oracle jdk8u281-b09
The object storage protocol requires the z/OS USS and object storage UTC date and time to match. Use the following USS command to display and verify the UTC date and time:
date -u
Note
If the UTC date and time do not match, the agent will fail with an HTTP error code 403 while trying to connect to the object storage.
The Model9 agent is invoked using the JZOS Batch Java Launcher which is a component of the IBM JVM for z/OS. To verify that the JZOS is correctly configured to run the Model9 agent, check that the JZOS JVMLDM86 module is located in your LINKLIST or use the STEPLIB DD.
Note
For more details about JZOS setup, refer to the JZOS Installation and User Guide: http://publibfp.dhe.ibm.com/epubs/pdf/ajvc0120.pdf
Create a dedicated ZFS filesystem for the Model9 Agent installation and mount point for the new ZFS, as demonstrated in the following JCL. Edit the JCL using the site’s standard naming convention for the mount point, ZFS and volume serial number if it is not managed by the SMS.
//M9AGTZFS JOB ACCT#,SYSPROG,TIME=NOLIMIT,REGION=0M,
// NOTIFY=&SYSUID,MSGLEVEL=(1,1),MSGCLASS=X
//* Create a Model9 installation directory
//CREATE EXEC PGM=BPXBATCH,
// PARM='SH mkdir -p /usr/lpp/model9'
//* Define and format a new zFS for Model9
//DEFINE EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DEFINE CLUSTER (NAME(SYS2.MODEL9.ZFS) -
VOLUMES(xxxxxx) -
LINEAR CYL(200 50))
/*
//FORMAT EXEC PGM=IOEAGFMT,REGION=0M,
// PARM=('-aggregate SYS2.MODEL9.ZFS -compat')
//SYSPRINT DD SYSOUT=*
//STDOUT DD SYSOUT=*
//STDERR DD SYSOUT=*
//* Mount the newly defined zFS to the Model9 installation directory
//MOUNT EXEC PGM=IKJEFT01,DYNAMNBR=10
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
MOUNT FILESYSTEM('SYS2.MODEL9.ZFS') +
MOUNTPOINT('/usr/lpp/model9') +
TYPE(ZFS) MODE(RDWR) /* AUTOMOVE */
//
Note
The parameters for the IOEAGFMT utility and the mount commands are case sensitive.
To automatically and permanently mount the ZFS after an IPL, add the mount command to the BPXPRMxx as shown in the following example:
MOUNT FILESYSTEM('SYS2.MODEL9.ZFS')
TYPE(ZFS)
MODE(RDWR)
MOUNTPOINT('/usr/lpp/model9') /* AUTOMOVE */
Note
For a Sysplex environment, remove the comment marks from the term AUTOMOVE in the JCL above to enable file system migration between Sysplex members.
Use an FTP utility to upload the Model9 agent’s installation tar file to the model9 directory created in the previous step. Use Passive Mode if supported by the FTP client. The tar file must be uploaded in binary mode as shown in the following example:
$ ftp mf-lp1
Connected to mf-lp1.
220-FTPD1 IBM FTP CS V2R2 at mf-lp1, 06:20:40 on 2017-02-23.
220 Connection will not timeout.
Name (mf-lp1:m9user): m9user
331 Send password please.
Password:
230 M9U is logged on. Working directory is "M9U.".
Remote system type is MVS.
ftp> cd /usr/lpp/model9/
250 HFS directory /usr/lpp/model9/ is the current working directory
ftp> bin
200 Representation type is Image
ftp> put model9-v2.8.2_build_ebcddfb-agent.tar
local: model9-v2.8.2_build_ebcddfb-agent.tar remote: model9-v2.8.2_build_ebcddfb-agent.tar
229 Entering Extended Passive Mode (|||1026|)
125 Storing data set /usr/lpp/model9/model9-v2.8.2_build_ebcddfb-agent.tar
250 Transfer completed successfully.
4483584 bytes sent in 00:02 (1.95 MiB/s)
ftp> quit
221 Quit command received. Goodbye.
Use the tar command in the z/OS UNIX shell to extract the agent files. During extraction, all agent files are saved in the same directory and can only be run by users who have the following permissions in the FACILITY class:
- BPX.FILEATTR.APF ACC(READ)
- BPX.FILEATTR.PROGCTL ACC(READ)
- BPX.FILEATTR.SHARELIB ACC(READ)
The directory name usually includes the agent’s release number. It is recommended to create an alias that does not include the release number for the directory. This makes future upgrades more transparent, as shown in the following example:
TSO OMVS
su
cd /usr/lpp/model9/
tar -xpf model9-v2.8.2_build_ebcddfb-agent.tar
# define the alias name "agent"
ln -s model9-v2.8.2_build_ebcddfb-agent agent
Create and copy the Model9 sample configuration directory using the following commands:
cd /usr/lpp/model9
mkdir conf
cp agent/sampleConf/* conf/
The Model9 z/OS UNIX files backup uses PAX. This functionality requires the creation of a program-controlled version of PAX. Use the following commands:
cd /usr/lpp/model9
mkdir bin
cp -p /bin/pax bin
# Set program-control flag
extattr +p bin/pax
The Model9 CLI (Command Line Interface) provides an interface for issuing Model9 commands from TSO / JCL. Install the feature by creating a listener directory under the agent’s main path:
cd /usr/lpp/model9/
mkdir listener
chmod 777 listener
chmod +t listener
Edit the JCL CPY#PDS located in
/usr/lpp/model9/agent/installation/ to create the Model9 LOADLIB, SAMPLIB, and EXEC PDS files. Submit the JCL.Customize the M9CLI rexx in the EXEC PDS to match installation standards:
fifodir = "/usr/lpp/model9/listener"
loaddir = "SYS2.MODEL9.V282.LOADLIB"
Note
Copy the M9CLI EXEC to your site’s local EXEC library.
Use the sample JCL M9USERST in the SAMPLIB PDS to define the security settings required by the agent.
Note
The M9USERST JCL permits discrete profiles. If your site uses generic profiles, update the JCL accordingly. If your site uses unprotected resources, no further action is needed.
Check you security product database to validate the type of resources being used.
By default, the M9USERST JCL contains a definition for the M9USER with the OPERATIONS attribute. Model9 also supports the RACF TRUSTED attribute in cases extended auditing is required. To perform this change follow the steps below:
- 1.Remove
OPERATIONSfromADDUSER M9USERcommand in the M9USERST JCL. - 2.Add
TRUSTED(YES)attribute to allRDEFINE STARTEDcommands within the STDATA section, for example:RDEFINE STARTED M9AGENT.** STDATA(USER(M9USER) GROUP(SYS1) TRUSTED(YES)) - 3.Add an RDEFINE statement for the life cycle procedure as the TRUSTED attribute is only supported for Stated Tasks.
RDEFINE STARTED M9LIFECY.** STDATA(USER(M9USER) GROUP(SYS1) TRUSTED(YES))
Note
Review and update the JCL to match local site standards. The SHARED parameter is specified by default. The SHARED.IDS permission is required to run the JCL.
Submit the JCL and verify all the commands were performed, review the output in case of any issues.
Copy the sample JCL M9AGENT from the SAMPLIB PDS to a local PROCLIB member. Update the JCL:
Update | Description |
DD STEPLIB | Model9 installation LOADLIB |
PWD environment variable | Model9 agent installation path |
CONF_HOME environment variable* | Model9 agent configuration directory path |
* CONF_HOME can be used for activating more than one agent in the same LPAR. The parameter will allow the agents to use the same Model9 installation files and libraries but have a different configuration directory. The recommendation is to have one agent per LPAR and to have all agents in the same GRS-complex point to the same Model9 complex. However, additional agents in the same LPAR may be required if using a sub-plex, having both development and production environments, or pointing the agents to different cloud storage. CONF_HOME must precede the stdenv-main.sh statement. The following is a sample use of the parameter:
//STDENV DD *
export PWD=/usr/lpp/model9/agent
export CONF_HOME=$PWD/../conf
export ENV=agent
. $PWD/scripts/stdenv-main.sh
//
Note
Add the agent procedure to the system’s startup process.
Before starting the agent, update the following configuration files located in
/usr/lpp/model9/confParameter | Description | Required | Default |
JAVA_HOME | 64-bit java home path | Yes | None |
TIME_ZONE | Java time zone, e.g. America/New_York | Yes | None |
TCPIP_NAME | TCP/IP stack used by the agent * | No | TCPIP |
AGENT_MEMORY | Allocated Java heap space for the agent. 1 GB is the minimum value. | No | 1g |
LIFE_CYCLE_MEMORY | Allocated Java heap space for life cycle management. 256 MB is the minimum value. | No | 256m |
SAPI_MEMORY | Allocated Java heap space for the server API (M9SAPI). 56 MB is the minimum value. | No | 56m |
SAPI_NETWORK_DEBUG | Enable network level debug for server API (M9SAPI). Expected values: “true” | “false” | No | false |
SAPI_KEYRING_NAME | The SAF keyring name to be used as a truststore during server API certificate validation. if specified, the keyring will be searched at: &SYSUID/<KeyRingName> | No | None |
* If running in an environment with multiple TCP/IP stacks, use the DISPLAY TCPIP operator command to display the TCP/IP stack name as shown in the following example:
Parameter | Description | Required | Default |
port | The agent’s listening port for server HTTPS communication | No | 9999 |
keystore | An agent keystore in JKS format which contains only one key entry to be used by the agent for incoming HTTPS requests. See Optional: Using the organization's certification authority. | No | agent.jks |
keystore_password | The provided keystore password | No | agent |
truststore | An agent truststore in JKS format which the agent will use to trust incoming HTTPS requests. See Optional: Using the organization's certification authority. | No | keystore value |
truststore_password | The provided truststore password | No | keystore_password value |
Parameter | Description | Required | Value |
objstore.endpoint.url | The URL of the object storage (Should start with “http://” or “https://”) | yes | default: none |
objstore.endpoint.userid | Access key to object storage | yes | default: none |
objstore.endpoint.password | Secret key to object storage | yes | default: none |
objstore.endpoint.password_encrypted | Determine if the object storage password is obfuscated or not | no | default: false |
objstore.resources.container.name | Container/bucket name | yes | default: model9-data |
objstore.endpoint.api.id | The object storage API name | no | default: s3 Amazon AWS: aws-s3 Microsoft Azure: azureblob |
objstore.endpoint.api.s3.v4signatures | When using object storage that uses V4 signatures, set this parameter to ‘true’, in addition to api.id: s3 | no | default: false Cohesity: true HCP-CS: true |
objstore.endpoint.no.verify.ssl | When using the HTTPS protocol, set to ‘true’ to avoid SSL certificate verifications* | no | default: true |
objstore.endpoint.tier | The object storage tier to use for created blobs. This option should only be used when using AWS S3 or Azure Blob. none will not request a specific tier for blobs; leaving the tier selection up to the object storage.infrequent maps to AWS S3's "Infrequent Access" tier and Azure Blob's "Cool" tier. | no | Default: noneAccepted values: none/standard/infrequent |
objstore.endpoint.api.s3.calculatemd5 | Enable MD5 checksum calculation to be included in putObject requests. *This is required for Object Lock enabled buckets | no | Default: false
Accepted values: true/false |
* To enable trust validation on the object storage certificate, use the
objstore.endpoint.no.verify.ssl parameter and set it to false.The
agent.yml file may contain additional parameters, setting them will override the defaults:Parameter Name | Description | Default value |
objstore.endpoint.shared.archive.volume | Archived volume serial name | M9ARCH |
objstore.functionality.backup. threadpool.size | Number of parallel threads for backup and archive runs. Increasing this parameter requires verifying that sufficient java heap space is defined for the agent . | 20 |
load.balancing.group.name | A group of agents that can share the work of a single policy. | &SYSPLEX |
restore.progress.interval. percentage | Progress interval by percentage | 10 |
restore.progress.minimum.size.mb | The minimum size in MB of a backup/archive action at which point the Agent starts reporting recall/restore progress | 1048576 |
objstore.resources.complex.id | Defines the resources the agent can process. | “group-&SYSPLEX” |
lifecycle.deleteExpired.gdg.minimumDays | The minimum number of days that a data set will be kept after being archived. After the number of minimum days has passed, the data set will be checked whether it is eligible for deletion. | 7 |
lifecycle.deleteExpired.gdg.intervalDays | After the minimum number of days has passed, the data set will be checked whether it is eligible for deletion. The data set will be checked every X days. | 4 |
lifecycle.deleteexpired.cds.catalogexpiration.minimumdays | The minimum number of days that a catalog controlled cloud data set will be kept in storage. | 7 |
cli.functionality.backup.compression | The compression type used for CLI backup execution. | gzip |
cli.functionality.archive.compression | The compression type used for CLI archive execution. | gzip |
import.allocate_use_expdt_98000 | Whether or not to add the EXPDT(98000) to the tape allocations. For CA-1 import support change to: true | false |
import.allocate_label_type | Which label type should be used by the tape allocation. For CA-1 import support change to: blp | sl |
dcollect.output.unit | DCOLLECT UNIT | VIO |
dfdss.input_primary_cyl | DFDSS SYSIN primary size | 1 |
dfdss.input_secondary_cyl | DFDSS SYSIN secondary size | 10 |
dfdss.input_lrecl | DFDSS SYSIN LRECL | 80 |
dfdss.input_block_size | DFDSS SYSIN block size | 80 |
dfdss.input_bufno | DFDSS SYSIN BUFNO | 1 |
dfdss.input_unit | DFDSS SYSIN UNIT | VIO |
dfdss.output_primary_cyl | DFDSS SYSPRINT primary size | 1 |
dfdss.output_secondary_cyl | DFDSS SYSPRINT secondary size | 10 |
dfdss.output_lrecl | DFDSS SYSPRINT LRECL | 134 |
dfdss.output_block_size | DFDSS SYSPRINT block size | 1340 |
dfdss.output_bufno | DFDSS SYSPRINT BUFNO | 1 |
dfdss.output_unit | DFDSS SYSPRINT UNIT | VIO |
dfdss.optimize | Set the DFDSS OPTIMIZE value for data set backup and archive | 3 |
discovery.missing_property.message.level | The message level used for resource level discovery failures while fetching properties.
Accepted values:
SEVERE
WARNING
INFO | WARNING |
idcams.temp.unit | IDCAMS temporary unit | VIO |
xcf.functionality.enabled | Controls the agent's XCF listener which enables the auto-recall and CDS functionality | true |
volumeBackup_dfdss_warn_to_ok_msg_codes | Controls which DFDSS warning mesages will not affect the return code of the DFDSS operation | ADR411W ADR333W |
model9.dfdss.statements.backup.forcecp.enabled | Enables the use of FORCECP for data set level backup (CLI & Policy).
Accepted values:
true
false | true |
model9.dfdss.statements.backup.forcecp.days | The FORCECP parameter days value. Accepted values:
0-255 | 0 |
Note
If the target z/OS installation does not currently support
VIO, add the following parameters to the agent.yml file and change the unit configuration options appropriately: dcollect.output.unit, dfdss.input_unit, dfdss.output_unit and idcams.temp.unitCloud data set parameters
Parameter | Description | Default value |
|---|---|---|
cloud_datasets.supported_read_utility_name_pattern | Determines the enabled applications for cloud data set read usage | "(IEBGENR)|(IEBGENER)|(DSNUTILB)|(IEBCOPY)|(IDCAMS)" |
cloud_datasets.supported_write_utility_name_pattern | Determines the enabled applications for cloud data set write usage | "(IEBGENR)|(IEBGENER)|(DSNUTILB)|(IEBCOPY)|(IDCAMS)" |
Resource complex and load balancing parameters
Agents sharing the same resource complex can all interact with the same resources. Actions such as backup/archive/recall/restore/list are all dependent on the resource complex parameter. Agents not sharing the same resource complex will not be able to list/restore/recall backups/archives created on a different resource complex. By default, all agents in the same Sysplex share the same resource complex. To update the default value of the resource complex, add the parameter to your agent.yml configuration file with a new value. Within a resource complex, you can define one or more load-balancing groups. See the User and Administrator Guide for more details.
Note
An agent can only access resources backed up or archived to the complex ID defined in its configuration.
Model9 allows obfuscating the object storage secret key so it won't appear in clear text within the configuration files.
Use the obfuscated object storage secret key that was produced during the server installation. When using the obfuscated secret add the following setting to the agent's configuration file:
objstore.endpoint.password_encrypted: true
Update the license file at /usr/lpp/model9/conf/model9-licenses.txt. The file can accommodate several licenses, for example, the license of all the LPARS in the Sysplex and the Disaster Recovery site’s license.
- Each license should be written on a separate line.
- Comments should be preceded with a “#” sign.
When multiple licenses are specified in the file, the most fitting license will be used, according to the following rules:
- The CPUID correlates with the CPUID of the executing LPAR. This is mandatory.
- From the licenses that correlate with the CPUID of the executing LPAR, the one with the latest expiration date will be used.
The license details to be used will be displayed in the agent’s log and in the UI. In case the agent could not find a valid license, an error message will be displayed as a WTO. The full error message will be available in STDOUT.
By default, the agent accepts requests from trusted clients using a self-signed certificate generated by Model9. In a production environment, it is recommended to replace the certificate with a signed organizational CA certificate.
The Model9 Automatic recall intercept allows the recall of archived files from within z/OS in a transparent manner. To install the automatic recall intercept:
Update the PROGxx configuration by adding the following statements:
APF ADD DSNAME(SYS2.MODEL9.V282.LOADLIB) SMS
LPA ADD DSNAME(SYS2.MODEL9.V282.LOADLIB) MOD(ZM9CPTN)
LPA ADD DSNAME(SYS2.MODEL9.V282.LOADLIB) MOD(ZM9S26X)
EXIT ADD EXITNAME(ZM9P_S026) MOD(ZM9S26X) PARM('M9ARCH')
Apply the changes using the SET PROG=xx operator command from any console:
SET PROG=xx
Note
The intercept and exit must be loaded to the Dynamic LPA in order for them to function correctly, do not use MLPA or PLPA to load the modules.
Copy the M9HOOK and M9UNHOOK from the SAMPLIB PDS to a local PROCLIB member. Customize M9HOOK accordingly and activate it using the following command:
S M9HOOK
Note
Add the M9HOOK JCL to your standard IPL process.
If an existing Model9 intercept has been uninstalled, then a new copy of the ZM9CPTN module must be loaded to the Dynamic LPA using the following command:
SETPROG LPA ADD DSNAME=SYS2.MODEL9.V282.LOADLIB MOD=ZM9CPTN
When using another data management product together with Model9 Cloud Data Manager, add the following DD statement to the other product’s procedure to avoid collisions:
//ZM9$NORC DD DUMMY
Restart the address space after applying the DD.
The Model9 life cycle management JCL is responsible for automatically deleting data sets (archives, backups and imports) that have expired. A sample JCL,
M9LIFECY, can be found in the Model9 SAMPLIB PDS.- 1.Grant your site scheduler permission for surrogate access to the M9USER:RDEFINE SURROGAT M9USER.SUBMIT UACC(NONE)PERMIT M9USER.SUBMIT CLASS(SURROGAT) ID(<scheduler>) ACC(READ)SETR REFRESH RACLIST(SURROGAT)
- 2.Update the life cycle management JCL:UpdateValue descriptionDD STEPLIBModel9 installation LOADLIBPWD environment variableModel9 agent’s installation path
- 3.Submit the life cycle management JCL in the simulate mode: Set --simulate to ‘yes’. The life cycle management JCL will list all the data sets that would have been deleted without actually deleting them.
- 4.Submit the life cycle management JCL in the initial mode: Set --simulate to ‘no’ or remove. The life cycle management JCL will scan all archived data sets with expiration dates in the past - up until today’s date. The next time the life cycle will run, it will resume where the previous process ended.
The Model9 Life Cycle procedure,
M9LIFECY, can be set to use the RACF TRUSTED attribute. This will require defining the JCL as a started task. Follow this step to configure it:- 1.Replace the M9LIFECY's JOB card with:
//M9LIFECY PROC
Note
Life cycle management JCL should be scheduled daily via the local site scheduler.
Start the agent from any console by issuing the following command:
S M9AGENT
Verify that the agent was started successfully. The following messages should appear:
ZM91002I MODEL9 BACKUP AGENT VERSION 2.8.2 INITIALIZING
ZM91000I MODEL9 BACKUP AGENT INITIALIZED
The optional Remote Server API feature allows the running of Model9 policies from within z/OS using standard JCL. To enable the Remote Server API feature:
- 1.Copy the sample JCL PROC M9SAPI from the SAMPLIB PDS to a local PROCLIB file and edit the procedure according to the instructions in the file. The JCL M9SAPIJ sample can be used to run a policy.
- 2.The Remote Server API uses HTTPS to communicate with the server. If a trusted certificate is not defined for the server, set
VRFYCERT=NOin the M9SAPI procedure to skip certificate validations. - 3.If a valid certificate is defined, make sure the user running the
M9APIJhas the correct SAF keyring defined to enable validation of the server certificate. TheSAPI_KEYRING_NAMEsetting defined in the samplemodel9-stdenv.shconfiguration can be used to specify the keyring name.NoteCommunications between the Remote Server API and the Model9 Server are encrypted regardless of theVRFYCERTvalue. - 4.When verifying the certificate used by the Remote Server API, it must be defined for a Resource Access Control Facility (RACF) using a keyring.
ISPF allows the definition of only one Level 2 migration volume. The default migration volume is
MIGRAT. In order to use the Model9 automatic recall function, the default volume must be set to M9ARCH, using the ISPF Configuration utility. Failing to change the migration volume will result in the following errors in ISPF screen 3.4:Command | Situation | Resolution |
Browse / Edit / View | Automatic recall will be rightfully triggered but the screen will continue to display M9ARCH as the VOLSER, instead of the DASD. The message "Tape not supported" will appear on the top right of the ISPF screen. | Enter REFRESH on the command line. The screen will be refreshed, showing the correct VOLSER, and the file will be available. |
LISTC or performing a REXX | Automatic recall will be unnecessarily triggered and the data set will be recalled | Use TSO LISTC instead of typing LISTC next to the data set name |
Note
Performing this step will cause any other auto-recall software to suffer from these errors.
To change the default ISPF settings:
- 1.Start the ISPF Configuration utility (TSO ISPCCONF) and select the site defined in the configuration table.
- 2.Select Option 1 to modify the existing table.
- 3.Change the “Volume for Migrated Data Sets name” to M9ARCH - the default Model9 archive volume name.
- 4.Save the settings.
- 5.Review the newly created configuration table file.
Either create and install the SMP/E USERMOD containing the new configuration table or create a load module that resides on a shared library containing the ISPF settings.