Deployments can range from personal laptop usage to large enterprise environments. The installer allows both interactive and unattended installations. Key folders include Users for individual data, Jobs for temporary execution data, Shared Public for shared resources, and XMLDB for the database. Logs record job executions, authentication events, and errors. Tools like DbUtil allow backup/restore of data, pkgutil creates packages for application delivery, and regress enables test automation. Planning folder locations and maintenance is important for managing resources in an enterprise environment.
DevEX - reference for building teams, processes, and platforms
(ATS6-PLAT07) Managing AEP in an enterprise environment
1. (ATS6-PLAT07) Managing AEP in an
Enterprise Environment
Conrad Agramont
Senior Product Manager, Enterprise Technologies
Accelrys Enterprise Platform
conrad.Agramont@Accelrys.com
2. The information on the roadmap and future software development efforts are
intended to outline general product direction and should not be relied on in making
a purchasing decision.
5. Laptop/Desktop Workgroup Enterprise Cloud HPC
Key Benefit Personal Productivity Collaboration Applications &
Management
Flexibility Performance
Managed by Scientific User Team/Application Admin IT + Application Admin Scientific User/Vendor HPC IT
Deployment One install, works out
of the box
Some customization, 1-
few servers
Many Servers
Web Farm
Cloud Vendor Grid Schedulers,
specialized hardware
Configuration None to simple Simple Flexible and Complete Complete Complete
Security: Authentication File and Active
Directory
Active Directory Active Directory, SAML,
Kerberos
Local Credential
SAML/OAuth
LDAP/Kerberos/PAM
Service Level Agreement N/A N/A Business Operations Cloud Vendor Business Operations
Spectrum of Deployments
Customer Choice
11. Folder: Users
• Global Property: UserDir
• Protocol Author:
– Store data relevant to a given user
– Data is not intended to be shared or used by others
• Admins:
– Keep track of size and what’s located in there
– Plan your storage allocation and speed according to your
business needs and level of Service Level
– Provide guidance to Authors and Developers
12. Folder: Jobs
• Global Property: JobDir
• Protocol Author:
– Stores data while running a job and when completed
– Treat as a temporary store.
– Store important results in a more permanent location (e.g. SharePoint,
Corporate File Store, or ELN) using a Writer component, HTTP Connector, etc.
– Any files that are in top level job folder are treated as “Job Results” to report to
the client
• Admins:
– This folder can grow rapidly, so you should plan on purging content based on a
well communicated date range (e.g. delete items older than 2 weeks)
Username Job GUID
<Server Install>webjobscagramontppc4C3CB-1CF8-4B90-97E2-6F1D9DA89C33
13. Folder: Shared Public
• Global Property: SharedPublicDir
• Protocol Author:
– When using with a Web Farm/Load Balance deployment
– Store data that won’t be impacted when job folders are cleaned
• Admins:
– Replaces the use of “ScitegicRoot”
– Oftentimes developers would use “$(ScitegicRoot)/..” to get to
the install directory
14. Folder: Local Temp
• Global Properties (Both mean the same)
– LocalJobTempDirectory
– TempDir
• Protocol Author:
– Default, this is in the same folder as the job. If job is running on a cluster or grid,
this I/O will travel across the network.
– Can set this at the protocol level to change during a given job.
• Admins:
– Move the default location for all jobs away from the network location of jobs (if
applicable)
– Set location to a fast local volume
– Schedule Defrag maintenance on this volume periodically. Make sure to use
Maintenance Mode.
15. Folder: XMLDB
• Protocol Author:
– Doesn’t really care
• Admins:
– Backup each time you change something major in production or
before a migration
– Backup can be done using Admin Portal, DBUtil, or manually using
xcopy
– In production, changes should be very infrequent. Volume location
should favor disk read performance
– Use “Windows Scheduler” or linux “cron” to create a “dbutil –
backup”
16. Folder: Logs
• Protocol Author:
– Interested Party, but doesn’t have direct access
• Admins:
– Lots of information on usage, performance, connectivity, etc.
– Use Pipeline Pilot to build reports
– The logs/messages/*.log files do not require purging
• Log rotation to limit the growth to 10MB each with one backup file
• Previously growth limit was 1MB per log
– Three categories of Logs (discussed more in later slides)
17. Log of Job Execution Events
• <install>/logs/usage/PipelinePilot.log.
• It is a structured, delimited file with one line per
completed job.
• The information includes the name of the protocol
executed, the time and duration, the client user, and
which packages were employed.
• The information is best viewed via the Completed Jobs
page in the Admin Portal.
18. Log of User Authentication Events
• <install>/logs/usage/access.log.
• It is a structured, delimited file with one line per
authentication event.
• It includes initial user login events and revalidation of the
user credentials.
• The information includes the event time, the login name,
the client program type, the authentication result
(0=failed, 1=success, 2=reset) and the client IP address.
19. Error and Diagnostic
• <install>/logs/messages
• scitegicerror_NNN.log, NNN identifies a process name. Possible names are:
– cgiadmin
– cgifile
– cgicache
– cgihelp
– scisvr
– runprotocol
– regression
– dbutil
– client
– pkgutil
– apache
– aepmanager
• javaserver_NNN.log, NNN identifies a java application name. Possible names are:
– catalog
– derby
– platform
– securityfilter
– tomcat_shared
• Files in the java subfolder - Java server error dump files, named jvm_error_<process-id>.log
20. Example of Folder separation
•RAID 1 (Mirrored)
•OSC:
•RAID 6 (Fast, redundant)
•AEP Installation, logs, XMLDBD:
•RAID 0 (Fast, no redundancy)
•Local TempE:
•Fast Remote Storage (E.g. IBM GPFS)
•Jobs, SharedDirF:
•Fast Remote Storage (e.g. IBM GPFS)
•Users, PublicG:
22. Dbutil
• Command line
• Great for automation
• Tasks
– Backup/Restore XMLDB
– Backup/Restore Config
– Copy tab/user folders
– Purge tab/user versions
23. Packaging (pkgutil.exe)
• Creating deployable units
• File/Folder based
• Manifest in XML
(ATS6-DEV06) Using Packages for Enterprise Application Delivery
25. • AEP 9 Deployment Guide is key to your planning
• Plan your job folder according to usage
• Educate your Authors and Users about folder usage and your
maintenance plans
• Schedule and automate your maintenance
• Use Unattended Installation for repeatable deployments
• Use Packages for repeatable and manageable application
deployments
https://community.accelrys.com/groups/it-dev
Summary