Parallel session: data centres

Parallel session H:
Data centres
Chair: Guy Sudron

Please switch your mobile phones to silent
19:30
No fire alarms scheduled. In the event of an
alarm, please follow directions of NCC staff
Dinner (now full)
Entrance via Goldsmith Street
16:30 -
17:30
Birds of a feather sessions
15:20 -
16:00 Lightning talks

Jisc shared datacentres
Guy Sudron

Shared datacentres update
»The Jisc portfolio
»The Jisc frameworks
»Current status
12/04/2017 Jisc shared datacentres

Jisc’s role
»Procurement management and contract authority
»Investment to start up the datacentre
› Procurement costs
› Legal costs
› Connection of the datacentre to the Janet core
»Ongoing promotion & awareness to drive uptake
»Anchor tenant group facilitator
»The datacentres as core to a broader ecosystem of services

Shared datacentre (south)
»Provided byVIRTUS, located in Slough
»Connected to Janet at 2 x 400Gb
»Framework launched in September 2014

SDC(south) tenants
Anchor tenants Other tenants

Current status SDC(South)
3 Jisc datahalls, c240 racks = 2.4MW
60/40 split HPC vs Enterprise
Projects: eMedlab, MedBio

Our new shared datacentre
»Conversations with universities based in the north of England who
had visited Slough led to:
› Full OJEU procurement, led by Jisc
› Working with 4 anchor tenants
› Contract signed in September 2016

Shared datacentre (north)
»Available now
»Provided by aql from their
new DC5 in Leeds
»Connected to Janet
at 2x100G initially
› Network live on 30th March

SDC(north) tenants
Anchor tenants Other tenants

The Jisc shared datacentre model
»Jisc frameworks
› 5 + 5 +5
› Full OJEU procurement
»Designed for education and research
› Research and enterprise applications
› Flexibility and scalability
»Datacentres connected to the Janet core

The Jisc shared datacentre model (2)
»Commercial offering
› Very favourable rack pricing (regularly benchmarked)
› Contractual PUE
› Pass through price for power

Shared datacentre model (3)
»Flexible services
› Able to ‘mix and match’ racks of varying power density with no compromise
in terms of power or cooling
› Racks can be supplied and installed by the provider or by the customer
› Smarthands; storage and assembly space; office space etc.
› Flexible networking – layer3, layer2VLANs, dedicated wavelength

Lessons learnt
»Perceived barrier of location away from campus not an issue,
but does need support from senior management
»SDCs allow central IT to contribute to institution’s ‘green agenda’
and improve efficiency
»Don’t under estimate the need for re-architecting services
»Clear, transparent, and manageable costs
»Network latency not an issue, but architecture and
services need to be planned

Findings to date
Estates
DC capacity
Re develop
Release forT&L
Research
Shared
Institution
Attract funding
Datacentre
Flexibility
Back-up
Prime
Hybrid
Shared services
Savings
Volume
Connectivity
Procurement
Power
Soft benefits recognised
» Collaborative R&D
» Shared resource
» Shared learning
» Best practice (HPC)
» Desire for on-campus
research data
» Some have capacity
» Tech, people, RDM,
storage shared services
» Build v buy
» Latency/location
» Recognition that collaboration
(incl health) is important
» Shared infra = best use of public
monies
» VAT
» ‘herd’ of tenants to max savings
» Power, build and plant – similar costs
» Space NOT always at
a premium

jisc.ac.uk
Contact me
Guy Sudron
guy.sudron@jisc.ac.uk
01235 822292 | 07941 657127

Jisc Services andTheir Part in
Data Centre Strategies
SDC North

Outline of Presentation
»WhyThe University of Liverpool supported the project
»What we were seeking from SDC North
»What did we learn from the process
»Where does SDC North fit in with UoL’s overall strategy
»What other services would we like to see in conjunction
with SDC North?
12/04/2017 Jisc Services and Their Part in Data Centre Strategy

Why UoL Supported the Project
»Recognised that the growth of IT services could not be met entirely
with on prem. solutions
»The original SDC in Slough had been a success and delivered a
service for sector tailored for their needs
»Important that we had input from as many northern universities as
possible
»The Jisc/Janet brand is strong and would encourage use in
the sector

Project
»As part of a consortium it was clear that the group had more
bargaining powers than a single entity
»Universities have very diverse requirements and need
flexible contracts
»Having an offsite DC with solid Janet connections was a
major factor
»As a member of N8 it was important that we helped to support the
requirements of HPC as part of the bid

What Did We Learn FromThe Process
»We were aware of the amount of time needed to support this and it
was quite a commitment
»Jisc’s procurement team are first class.The complexity of the
procurement would have been too much for an individual university
»We were able to get all our requirements addressed.This was no
mean achievement given the range of options
»Being able to have a contract for a single rack to a full HPC
installation was impressive.

Plans
»The UoL’s data centre strategy covers local, colocation and
cloud services
»The colocation element is especially important. It is the one that
requires a robust agreement to meet university needs.
»Primarily we wanted a small presence for BC.The agreement allows
for this and we still benefit from reduced rates created by the larger
tenant groups
»The SDC is the basis for other shared services

See
»The presence of Jisc in the DC allows the development of other
shared services
»Potential for UoL are archiving services via the DC, connectivity to
Cloud providers, a storage service etc
»Options to create collaboration with other Universities.
»Access to data sets held locally at the DC seen as advantageous.
»Potential development of other HPC services

See
»Thank you for your attention – Any Questions?

jisc.ac.uk
Steve Aldridge
University of Liverpool

DataCentres
Professor Paul Jeffreys
The Institute of Cancer Research
12/04/2017All Aboard the Cloud Express

, Prof P.W.Jeffreys, ICR, Networkshop, April 2017
Data Centres
ICR’s Distributed Storage Platform
at the Jisc Shared Data Centre
Paul W Jeffreys
12 April 2017

29
Our mission is
to make the
discoveries that
defeat cancer.
Our mission is
to make the discoveries
that defeat cancer

Presentation Outline
1. ICR / RM - very much part of the story
2. Existing ICR Research Infrastructure
3. ICR Research Data Storage Programme
4. RDS Service: solution, rationale, design
5. Overview of RDS Service

1. Institute of Cancer Research
and
The Royal Marsden

ICR at a glance

Academic successes
33
The ICR is ranked as the top academic research centre
in the UK. cCame first in the Times Higher Education
league table of university research quality compiled from
the Research Excellence Framework (REF 2014)
Together with
The Royal
Marsden we
are rated in
the top four
cancer centres
globally
measured by
citation impact
Joint top of the Times Higher Education table for
Innovation – based on worldwide citation of our
research in patents.
Joint top of the Times Higher Education table for
Innovation – based on worldwide citation of our
research in patents.

Our substantial and diverse funding
34
2016
• Total income £162m
• HEFCE 12% based on research excellence
• Grant income 38%
• Legacies and donations 7%
• Invention income from our discoveries 16%
Total incoming resources 2016
12%
Higher Education
Funding
Council for England
38%
Research grants
16%
Royalty income
7%
Legacies and donations
3%
Investment and tuition fees
24%
Sale of part of our future
royalty stream

Royal Marsden Hospital
35
The ICR’s unique partnership with The Royal Marsden and
“bench-to-bedside and back” approach means we make
discoveries and deliver clinical impact in a unique way
We have an outstanding record of research
achievement dating back more than 100 years
Together we are rated in the top
four comprehensive cancer
centres worldwide for impact on
cancer research and treatment

ICR and Royal Marsden Vision
Dynamically adapt treatment to individual patient over time and in response
Requires: data integration and Big Data analytics

Making the discoveries
Our strategy to defeat cancer
37
The ICR and The Royal Marsden
have worked together on a joint
strategy covering the next five years.
Our vision
We will overcome the challenges
posed by cancer’s complexity,
adaptability and evolution through
scientific and clinical excellence,
innovation and partnership

Making the discoveries
The four pillars of our strategy
38
2
3
Unravelling cancer’s complexity
We will comprehend the full complexity of cancer by harnessing the power of new
technologies and Big data
4
Innovative approaches
We will take on the challenge of cancer’s complexity, evolution and drug resistance through
the discovery of innovative new approaches to cancer treatment
Smarter, kinder treatments
We will move a step closer to cure by bringing personalised treatments into the clinic and
developing them for patients
Making it count
We will deliver better outcomes and improved quality of life for patients by establishing
innovative treatments, diagnostics and strategies for prevention as part of routine
healthcare
1

Two sites in London:
Sutton and Chelsea
39
Chelsea
39
Sir Richard Doll
Building
Opened 2006
Centre for Cancer
Imaging
Opened 2015
Brookes Lawley
Building
Opened in 2003
Centre for
Molecular
Pathology
Opened 2012
Chester Beatty
Laboratories
Fulham Road
Sutton

ICR Research Divisions
Breast
Cancer
Research
Cancer
Biology
Cancer
Therapeutics
Clinical
Studies
Molecular
Pathology
Radiotherapy
and Imaging
Structural
Biology
Genetics and
Epidemiology

The London Cancer Hub
A global centre for cancer innovation
41
We plan to deliver an exceptional
environment for cancer research that
enhances the discovery of new treatments
and their development for patients.
We will provide state-of-the-art facilities, and be
joined by a multitude of high-tech enterprises in a
network of 10,000 researchers, clinical staff and
support staff all operating from one site.
The London Cancer Hub aims to create a world-
leading life-science campus specialising in
cancer research, diagnosis, treatment, education
and biotech innovation.

2. Existing ICR Research Infrastructure

Existing Research Infrastructure:
End-of-life Data Storage Service
• NGS – 90%
HPC Cluster
• At Jisc Shared Data Centre: 1080 cores c. 38GFlops
• Similar capacity in Chelsea/Slough
• NGS – 70%
• 70% average use
Jisc Shared Data Centre:
• Most of HPC resource will be based in Jisc data centre at Slough
• No remaining space and power in London
Implementing new ICR Research Data Storage (RDS) Service

3. ICR RDS Programme

ICR RDS Programme
Bridge
Tier1 Tier2

RDS Service User Requirements
Collected from researcher workshops:
• 6PiB* storage, expandable to growing research need
• High level redundancy ensuring robust solution
• Cost effective and competitively priced solution
• Rapid access to all levels of data
• Ability to ‘snapshot’ to protect against accidental loss of datasets
• Ability for researchers to manage data transfers between Tiers
• Staging Areas to guarantee access to storage from instruments
* PiB is a binary unit of storage equivalent to 1.13PB (the decimal version)

Research Data Growth
“Imaging instrumentation is currently in a period of rapid technological
advance. It is therefore difficult to project the likely data acquisition volumes
beyond a relatively short predictive horizon (for example 6 – 9 months)”

RDS Service Decisions
New type of leading, innovative, cost-effective service for ICR
Decisions made in partnership with researchers:
• Single service for research data under active use
• Service with two tiers (best and most cost-effective)
• Choice of solution / supplier
• Replication / resilience
• Expandability / flexibility
• Disk only
• No “backup”
• Single namespace; straight forward to use

4. RDS Service
solution, rationale, design

RDS Service Schedule
24 Mar 2016: Invitation to Tender (ITT) published
29 April 2016: ITT exercise complete
• Different types of solution offered
• Best and final offers requested
31 May 2016: Preferred supplier chosen
• OCF / DDN
• GPFS and Object Store
• Over three locations
12 Aug 2016: Contract signed with OCF/DDN
7 Dec 2016: Service delivered to users (Tier 1)

RDS Service – Location
Slough
ChelseaSutton
Geographically distributed system to increase level of
protection for ICR data
Service located in Sutton and two different data halls
in Slough
10Gbps private network between sites
Each site also has a 10Gbps JANET connection to
the internet
Network traffic can route around the failure of any
inter-site link

RDS Service Network

RDS Service – High Level Architecture

RDS Service – Conceptual Design

RDS Service - Principles
Tiered and disk-only system:
Tier 1: mirrored, fast storage for instant access to
files by large numbers of clients
Tier 2: less performant, but more scalable object
storage for large volumes of data; less frequently
used data will mostly be on Tier 2
Able to offer direct access to both tiers from
desktops, servers and HPC
To the end-user it looks like one big “bucket” of
storage – complexity is hidden

RDS Tier 1 Characteristics
• Located at Jisc Shared Data Centre (Slough) and Sutton
• Two Gridscaler 7ks + GPFS Tiebreaker
• Allows failover to surviving site without disruption
• Stretched (Mirrored) GPFS parallel filesystem with single namespace
• SSD cache
• 8TB HDDs
• MEDIAScaler provides clustered NFS and SMB
• 4 NAS* gateway servers serving as GPFS nodes (2 Sutton, 2 Slough)
• 2PiB usable capacity (really 4PiB)
• No SPFs
* Growing to 8

RDS WOS Bridge Characteristics
• Based on HSM rules
• In conjunction with user, over ridable rules
• Selected files and directories will be migrated to the long-term storage
component
• Bridge rules for each research team (in principle)
• WOS Bridge uses synchronization service to exchange name space and data
changes between Tier 1 and Tier 2 objects that have been migrated to the
long-term storage
• Migrated file data is protected and replicated according to the target WOS
policy of the WOS Bridge rules

RDS Tier 2 Characteristics
• Located at Jisc Shared Data Centre (Slough) in two data halls, and Sutton
• Based on DDN Object Storage system
• 3-site WOS cluster based on 12 x WOS9660 storage appliances
• 4 per site
• 12 x WOS9660 appliances communicate over standard TCP/IP and expose a
single global namespace
• Global ObjectAssure used (erasure coding across 3 sites)
• Hierarchical erasure-coding scheme encoding object
• Expansion factor – 1.875 times
• Designed for cost-effective resiliency
• Allows a site offline + additional 2 disk failures in the remaining appliances
without impacting data availability
• 4.3PiB usable capacity (really 8PiB)

RDS Service – WOS Bridge Example
Example data flow:
1. Copy to Sutton T1
2. Mirror to Slough T1
3. File not used for a Month
4. Create on object
5. Stub files
File
Object
File FileStubStub

Current Use of Tier 1
0
100
200
300
400
500
600
700
RDS Data Storage Use (TB) by Date
Total Use

5. Overview of RDS Service

RDS Service Status
RDS
• Tier 1 - live 7 December 2016
• Tier 2 - operating from March 2017
• Bridge in process of being switched on, team by team
• Daily snapshots being taken
• Service has run without interruption
Decision to use Jisc Shared Data Centre
• No remaining room nor power in London
• Needed geographical separation
• Used data halls 1 and 3 (for Global ObjectAssure)
• Reliable, resilient, well managed data centre
Intention to grow RDS Service, add other elements of RDS programme

Thank You
and
Questions
paul.jeffreys@icr.ac.uk

Cancer Research Centre of Excellence
An ICR and Imperial partnership
12
• The ICR and Imperial College London have
developed a strategic partnership through the
creation of a virtual
Cancer Research Centre of Excellence.
• The partnership will harness the complementary
expertise of both partners to deliver an
enhanced and synergistically world-leading
programme of cancer research.
• Pooling resources and expertise, and working
together in strategic partnership, will enhance
the ICR’s basic and translational research.
ICR scientists will gain
access to the extensive
resources and expertise of a
multi-faculty university in new
areas, including:
• basic life sciences
• medicine
• physical sciences
• bio-engineering

Parallel session: data centres

Recommended

Recommended

More Related Content

More from Jisc

More from Jisc (20)

Recently uploaded

Recently uploaded (20)

Parallel session: data centres

Editor's Notes