DSPy a system for AI to Write Prompts and Do Fine Tuning
Complete Presentation | Federal Mobile Computing Summit | July 9, 2013
1. Welcome to the 2013
Federal Mobile
Computing Summit
#mobilefeds
2. Host Organization
Advanced Mobility Academic
Research Center
AMARC is a non-profit organization that focuses on
the three areas: Academic, Government & Corporate.
The Academic Research sector is the bridge between
Government and Corporate participation.
www.amarcedu.org
3. The Federal Mobile Computing
Summit Mobile App is now
available for download
#mobilefeds
4. Agenda
8 to 9 a.m. – Opening of Mobile
Technology Showcase
9 to 9:15 a.m. – Welcome, Dr. Rick
Holgate, ATF
#mobilefeds
5. Agenda
9:15 to 10:00 a.m. – Panel 1, Mobility
in the Federal Government, What’s
Next?
Moderator: Dr. Rick Holgate, ATF
Panelists: Kevin Cox, DOJ; Robert
Palmer, DHS; Mark Norton, DOD
#mobilefeds
6. Mobility in the Federal Government –
What’s Next?
July 9, 2013
7. 7
Federal Digital Government Strategy (DGS) Milestones of Interest
DGS Milestone 9.1
Use Cases and Top Challenges
DoD, DHS, DOJ, and NIST Mobility Efforts
Moving Forward
Agenda
8. 8
Milestones of Interest
Information Centric
MS 1.2 Open Data web-based availability
MS 2.1 and 2.2 High Value Data web based availability
Shared Platform
MS 3.3 BYOD
MS 3.6 Shared Mobile Application Development
MS 4.2 Development and delivery of digital services
MS 5.1 Wireless Federal Strategic Sourcing Initiative (FSSI)
MS 5.2 Enterprise-wide inventory (CMDs and Contracts)
MS 5.3 Analysis of enterprise contract vehicles
MS 5.4 Delivery of mobile apps
MS 5.5 Government-wide Mobile Device Management
Customer Centric
MS 6, 7 & 8 Customer focus & satisfaction
Security and Privacy
MS 9.1 Mobile Security Baseline and Mobile Security Architecture
MS 10.2 Accelerate mobile technology adoption
MS 10.3 Standard Approach to PII
9. Federal Mobile Security Baseline and
Reference Architecture
9
DELIVERABLE
Milestone 9.1 Federal Mobile
Security Baseline
DELIVERABLE
Mobile Security Reference
Architecture (Appendix: Mobile
Computing Decision Framework)
10. Government Mobile and Wireless Security Baseline – A use case driven
security approach for four challenge areas identified in gap analysis of MS
10.2
Mobile Device Management (MDM) (COMPLETED, based on NIST SP
800-53 rev4 and DoD MDM SRG)
Mobile Application Management (MAM) (COMPLETED, based on NIST SP
800-53 rev4 and DoD MDM SRG)
Identity and Access Management (Notional)
Data Sharing Standards (Notional)
Mobile Security Reference Architecture (COMPLETED)
Will be developed into a service level architecture by September
Mobile Computing Decision Framework (COMPLETED)
Federal Digital Government Strategy
Milestone 9.1
10
12. DoD Mobile Device Strategy, 8 Jun 2012
DoD CMD Implementation Plan, 15 Feb 2013
Mobility Inventory Memo, 15 Mar 2013
CMD Pilot Consolidation Memo, 21 Mar 2013
Mobility BCA Memo, 15 Apr 2013
Mobility STIGs (iOS, Android, BB), May 2013
DMCC devices provisioned, May 2013
DISA MDM/MAS RFP awarded, 27 Jun 2013
NIAP Mobile Protection Profiles, CY 2013, Q3
Key DoD Mobility Efforts
12
13. Guidance Documents
NIST FIPS 201-1: Personal Identity Verification (PIV) of Federal Employees and Contractors
NIST FIPS 201-2 (DRAFT): Personal Identity Verification (PIV) of Federal Employees and Contractors
NIST SP 800-46 R2 (DRAFT TBD): Guide to Enterprise Telework, Remote Access, and BYOD Security*
NIST SP 800-53 R4: Security and Privacy Controls for Federal Information Systems and Organizations
NIST SP 800-73-4, Part 1 (DRAFT): PIV Card Application Namespace, Data Model and Representation
NIST SP 800-73-4, Part 2 (DRAFT): PIV Card Application Card Command Interface
NIST SP 800-73-4, Part 3 (DRAFT): PIV Client Application Programming Interface
NIST SP 800-114 R1 (DRAFT TBD): User's Guide to Telework and Bring Your Own Device (BYOD) Security*
NIST SP 800-124 R1: Guidelines for Managing and Securing Mobile Devices in the Enterprise
NIST SP 800-157 (DRAFT TBD): Guidelines for Personal Identity Verification (PIV) Derived Credentials*
NIST SP 800-163 (DRAFT TBD): Guidelines for Testing and Vetting Mobile Apps
NIST SP 800-164 (DRAFT): Guidelines on Hardware-Rooted Security in Mobile Devices
Key NIST Mobility Efforts
13
Additional Efforts
– Collaboration with DARPA on the TransApp Program
– Collaboration with NSA on the Enduring Security Framework (ESF)
* Taken from csrc.nist.gov/documents/nist-mobile-security-report.pdf
14. Key DHS Mobility Efforts
14
DHS Mobility Strategy (DRAFT)
DHS Mobility Implementation Plan
(DRAFT)
Mobile Pilot(s) Consolidation Memos
(DRAFT)
WorkPlace as a Service (WPaaS)
Mobile Container solution,
APR 2013
Mobile Application Vetting Platform
(CarWash)
Initial Proof of Concept, MAR
2013
DHS Enterprise Wireless Contract
Awarded APR 2013
Information Resource Management
Strategic Plan (DRAFT)
FED Initiatives
DGS
Changed the way we look at delivery of IT
services and data
DHS Initiatives
WorkPlace Transformation
Changing Business Model
Maintain Federal Relationships
Application Lifecycle Management (ALM)
From concept to O&M
Code Standards, Sharing, Testing Standards,
Drive Tool development, Distribution
Models, Context at the Presentation
Accomplishments
15. FOCUS: DHS Mobility Efforts
15
Data Standards
Structure, Tagging, Labeling, Temporal value
Remove Context
Aggregation Issues
Authentication and Authorization
Form Factor Issue, Device, App, or Network Level?
Legacy infrastructure
Mission Partners
Application Services
Move to data layer protection
What to do in the interim?
Progression of the trust level…
Infrastructure
Concept of Internet as transport
Capability to support increased communications
16. Mobility Strategy
Partnerships with other Departments/Agencies
App Development Strategy
Pilot of New Handheld Devices
Tablet Pilots
Standard Tablets
Hybrids
BYOD Pilot
Key DoJ Mobility Efforts
16
17. Building on the Digital Government Strategy Mobility Milestones
Mobile Identity Management
Mobile Application Development and Vetting
Federal Mobility Solutions Architecture
Moving Forward
17
18. Federal CIO Council Committees
New Innovation Committee
Information Security and Identity Management Committee (ISIMC)
CIO Council will help coordinate Interagency efforts, including involvement
with OMB, GSA, and NIST
Federal Digital Government Strategy
Continue to build on Baseline and Reference Architecture (Milestone
9.1), as well as on other completed Milestones
Partnered with NSA for security
GSA: future contracts for Mobile Device Manager, mobile devices and
Airtime/Data Plans
Mobile Applications Reciprocity across Agencies
Expedite Mobile Security Approvals
Collaborative Technology Exploration and Standards Development
Building on the Digital Government
Strategy Mobility Milestones
18
20. DHS – Mobile Application Continuous Integration Orchestration Platform and
Mobile CoE, aka “CarWash”
NIST/DARPA – TransApp Program (NIST SP800-163 DRAFT Coming Soon)
DoD
Software Assurance in Defense Acquisition Guidance
DISA Mobility PMO – Mobile Applications Security Requirements Guide
(SRG)
NSA Information Assurance Directorate (IAD) Center for Assured Software
(CAS)
GSA Mobile PMO and Digital Services Innovation Center – Mobile Application
Development Program
Mobile Application Development and
Vetting
20
21. Builds on the Federal Mobility Reference Architecture
Building on Department/Agency (D/A) Use Cases
Utilizing information gathered from D/As during development of
DGS Milestone 10.2 (Mobility Barriers/Opportunities/Gaps)
Federal Mobility Solutions
Architecture
21
22. Programs/Opportunities
GSA FSSI Wireless: Wireless Federal Strategic Sourcing Initiative BPAs
GSA Mobility Management Solutions: potential MDM/MAM solutions sources
DGS Milestone 3.6: GSA Mobile Application Development Program
U.S. Government APIs: API Developer Resources
NSA CSfC Program: Commercial Solutions for Classified Program
DISA BAA 12-01: Mobile Device Common-Access-Card-Enabled Virtual Thin Client
Working Groups
ICAM Subcommittee (ICAMSC) Working Groups: various [e.g., CNSS IdAM WG, Logical Access
WG]
DoD Commercial Mobile Device Working Group (CMDWG) – Next meeting 19 Sept 2013
DoD PKE Mobility TIM –next meeting tentatively Dec 2013
References
HSPD-12: Policy for a Common Identification Standard for Federal Employees and Contractors
OMB M-11-11: Continued Implementation of HSPD 12
DTM 08-006: DoD Implementation of Homeland Security Presidential Directive - 12 (HSPD-12)
NIAP CC Protection Profiles (PP): various [e.g., Mobile OS, VoIP Apps, WLAN]
DISA SRGs: various [e.g., MDM, Mobile Policy, Mobile App, Mobile OS]
More Information
22
23. Agenda
10:00 to 10:45 a.m. – Panel 2, Use
Cases of Mobility
Moderator: Dave Perera,
FierceGovernmentIT
Panelists: Pam Hird, USDA; Dr.
David Rogers, University of Central
Florida
#mobilefeds
24. Federal Mobile Computing Summit
Washington DC July 9
Use Cases of Mobility
Pam Hird, CAPI Program Manager
United States Department of Agriculture
National Agricultural Statistics Service
Pam.hird@nass.usda.gov
202-615-9845
25. USDA - NASS Office
CAPI Solution
Field Interviewer
Apple iPad 2, 3, 4
3G/4G & WiFi Enabled
Cell Tower
USDA-NASS
Web Survey Site
Electronic Data Repository
(EDR) Wireless
Broadband
Agricultural Operation
Information Cloud
Questionnaires
Transmitted to Device
Data Saved for
Review to Cloud
Data Submitted to
the Office
Data
entered
into device
Fully automated
Flexible
Scalable
26. Unique Solutions
Apple platform
No data stored on iPad
Cloud Technology
Electronic Data Repository (EDR)
o Rewritten
o Non platform specific
AJAX
All pieces of the solution can be replaced
without changing the process
27. Application Solution
Electronic Data Repository (EDR)
o Original written in response to Paperless Act
o Non platform specific
o Rewritten to include Ajax and improve speed
o Custom designed for NASS data collection
JavaScript and DoJo
Inclusion of AJAX for transmission
Work in progress – evolving based on need
28. Built in Efficiencies
Electronic coordination of multiple surveys for respondent
Question coordination – duplicate questions prefilled
Electronic enumerator assignment
Mapping feature (plan route)
Timeliness of Data (48 – 72 hours)
Supervisor management interface
• Reassignments electronically
• Remote review questionnaire simultaneously
Quality checks and training tool
Soft Edits, Sum Fields, Skip Logic
52. Agenda
10:45 to 11:05 a.m. – Results of
FierceGovernmentIT & Market
Connections Pulse Poll on Government
Mobility
Speakers: David Perera,
FierceGovernmentIT; Monica Mayk
Parham, Market Connections
#mobilefeds
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64. Agenda
11:05 to 11:30 a.m. – Visionary
Keynote – Walter Johnston, FCC
11:30 to 12:00 p.m.
Mobile Technology Showcase
#mobilefeds
66. Measuring Broadband America
• FCC National Broadband Plan
– Released 3/10
– Advocated FCC undertake measurement of Broadband
America in collaboration with industry
• FCC fixed broadband measurement program
– Reports released 8/11, 7/12, 2/13
– Ongoing: yearly reports
– Generally perceived as most accurate measurements
of consumer broadband performance
• Result of transparency of process and collaboration with
industry and other parties
67. mobile Measuring Broadband America
• On 9/21/13 FCC announced extension of
program to mobile services
– Collaborative effort with AT&T, Sprint, T-Mobile,
Verizon, CTIA and others
• Data collection effort to start later this year
with initial public results expected early 2014
68. What we are doing
• Smartphone app runs in background on
volunteers phone
– Runs automatically at random times performing
measurements
– App provides only network related data and
results annonymized
• Data uploaded to cloud for presentation in
maps and other analysis
69. What We Are Collecting
• Network Performance Data, e.g.:
– Upload/Download Speed
– Packet Loss
– Latency
• Network Structure Data, e.g.:
– Service Coverage Data
– Tower ID/Signal Strength
– Bearer Channel
70. What is Unique About the Program
• Core foundation: consumer privacy
• Collaboration with key industry players
• Transparency in process
• Statistically valid data collection
• Open, reproducible and extendable
methodologies
71. Consumer Privacy
• Central to FCC’s policies and concerns
– Mobile data raises special issues
• Extensive discussions with representatives
from Federal Trade Commission, FCC, service
providers, academics and others to develop
formal privacy policy
• Mechanisms in place to ensure data is
anonymized and processed in manner to
ensure anonymity
72. Industry Collaboration
• Key tenant of National Broadband Program to
work collaboratively with major stakeholders
– Increases accuracy and success of program
– Provides potential for industry standardization
• Working with four major wireless carriers,
CTIA and others in open process
– Industry beta testing client software
73. Transparency
• Open meetings to discuss program milestones
• Publication of testing procedures, statistical
scripts, measurement methods, etc.
• Client software provided as open source
• FCC plans to provide GIS database displaying key
metrics
– Underlying data freely available (subject to consumer
privacy policies)
• Future goals include making available thru FTP and through
API interfaces on FCC database
74. Statistically Based Data Collection
• Android client software runs autonomously in
background
– Tests performed regularly on randomized basis
– User bias minimized/eliminated
• Avoids user testing to determine ‘what’s broken’
• Collaboration with industry permits more
targeted and more successful volunteer
solicitations
75. Long Term Objectives
• Continue to explore further utility of data
– We’ve focused on ‘low hanging fruit’
– Ongoing discussions with parties as to potential
uses
• Continue to provide accurate data on
evolution of broadband
• Promote standard metrics and cost efficient
data collection mechanisms
• Continue successful collaborative model
76. Agenda
12 to 1 p.m. – Mobile Innovation Awards
Presentor: Tom Suder, Advance Mobility Academic
Research Center (AMARC)
The Mobile Innovation Awards recognize those individuals
that use mobile solutions to better accomplish their
agency’s mission
Awardees: Gray Brooks, GSA; Kevin Cox, DOJ; Brad Nix,
USDA; Dan Rieken, WHCA
#mobilefeds
77. Agenda
1 to 2 p.m. – Mobile Technology Showcase
2 to 3 p.m. – Wrap-up of MITRE Mobile
Collaboration Sessions
Moderator: Pat Benito, MITRE
Panelists: Robert Anderson, U.S. Marine
Corps; Karen Britton, EEOP; Jim Quinn, DHS;
Harvey Reed, MITRE
#mobilefeds
78. Agenda
MITRE Mobile Collaboration Sessions
Challenge Areas
Challenge Area 1: Identity, Credentialing &
Access Management (ICAM) Using Mobile
Devices
Challenge Area 2: Contextually Aware
Mobile Applications
Challenge Area 3: Mobile Cyber Strategy
Challenge Area 4: Acquisition Challenges for
Acquiring Mobility
#mobilefeds
80. Agenda
3:00 to 3:45 p.m. – Panel 4, Mobile Security
– What The Research Says About Current &
Future Threats
Moderator: Dan Mintz, Advanced Mobility
Academic Research Center (AMARC)
Panelists: Kim Hancher, EEOC; Michael
Robinson, Stevenson University; Angelos
Stavrou, George Mason University; Nan
Zhang, George Washington University
#mobilefeds
81. Agenda
3:45 to 4:15 p.m. – Afternoon
Visionary Keynote, Marilyn Rose,
DHS
#mobilefeds
83. 83
Background
•Presidential directive for a comprehensive
Digital Government Strategy (DGS)
–Mobile Security Reference Architecture (MSRA) - Part A
–Mobile security baseline controls
–Mobile Computing Decision Framework (MCDF)
•Released to agencies in May 2013
83
84. 84
Part B Objectives
• Develop a Mobile Security Solutions Reference
Architecture – Part B for Federal Civilian Executive
Branch Departments and Agencies.
• Derive from use case based solutions that will allow
agencies to incorporate security and privacy into their
networks
• Take into account differing agency mission types.
84
85. Department/Agency Only Workshop
85
Department/Agency Problem Challenge Workshop – July 16
Location: Software Engineering Institute, 4401 Wilson Blvd.,
Arlington, VA
Engage D/As and discuss current implementation maturity levels,
solutions and struggles with their mobile implementations.
Develop use case based scenarios to be presented at the Vendor
Problem Challenge Workshop.
86. Vendor Only Workshop
86
Vendor Problem Challenge Workshop – July 25
Location: NRECA Conference Facility, 4301 Wilson Blvd.,
Arlington, VA
Registration is required
Limited to 80 participants
Present and discuss with vendors the use case based scenarios
developed from the D/A Problem Challenge Workshop.
Encourage vendors to provide solid and viable solutions for D/As
regarding mobile security issues.