2. 2
SchoolsFirst Federal Credit Union
• Started in 1934
• 645,000+ Members, consist of school employees and their
families
• Can serve Members in 10 Southern California counties
• High touch service (like Nordstrom, Zappos, etc.)
• $11+ billion in assets
• 5th largest credit union in the US and largest in California
• Online channels growth increasing faster than traditional
channels
3. 3
My Role @ SchoolsFirst FCU
At SchoolsFirst FCU since May 2013
Manager of Infrastructure Operations @ SFCU:
– Joined May 2013 as Manager, Infrastructure and Application Support
– Lead 20 person team supporting:
– Contact Center, Network and Telecom
– Compute and Storage
– Business Applications
Responsible for monitoring strategy
Splunk user for 2+ years
4. 4
My Background
20+ years in IT Management
– IT Ops, Infrastructure and Security
Employers
– SchoolsFirst Federal Credit Union
– Cisco Systems
– NDS – Pay TV solutions
Conditional Access/Digital Rights Management 125 million pay TV
households, Middleware 214 million devices, DVR 47 million
5. 5
Before and After with Splunk
… and more
Logs easily searchable
Proactive support
Lower MTTR, some prevention
Alerts and dashboards
High value from logs
Logs difficult to search
Reactive support
High MTTR
Limited visibility
Low value from logs
6. 6
Initial Use Case – Online Banking
… and more
70+ hosts
AIX, Windows, Linux, MS SQL, IBM stack (Websphere, HIS, EAI), network, etc.
Over 500 log files
~10 million calls per day
Urgent need to reduce outages
100 GB license
7. 7
Architecture and Data Sources
Wire Data Router Load Balance Firewall Applications VM’sActive Directory AIXRed Hat WindowsSwitch Telephony/
Contact Center
Data Sources
Other
Indexers
Search Simple Dashboards Users Admins Custom Dashboards Alerts
Deployment
Server
Search Head
License Growth
12/2013 - 100 GB
4/2014 - 175 GB
10/2014 - 375 GB
3 Splunk Admins
~10 active users
8. 8
Dashboard - Online Banking
• Splunk correlates impact across related systems during an outage
9. 9
Dashboard – Core Banking
• Splunk brings visibility into legacy core banking system
13. 13
Lessons Learned
Know your data
Follow standards - Common Information Model
Architect and test for scalability
Don’t skimp on professional services or training
Don’t hesitate to ask for help from Splunk internal resources
Build out an application on-boarding process
14. 14
What’s Next
Expand user base outside of IT Ops
Build center of excellence
Access dashboards with Splunk Mobile App
Enhance integration between Splunk and other tools
Expand coverage to Dev/Test and DR environments
Focus on KPI’s vs metrics
Editor's Notes
So my other areas of responsibility include all of the infrastructure. So storage, compute, network, telephony, contact center about a hundred back up as applications.
So my other areas of responsibility include all of the infrastructure. So storage, compute, network, telephony, contact center about a hundred back up as applications.