State of GeoServer 2.12

Andrea Aime
Technical Lead
andrea.aime@geo-solutions.it
@geowolf
GeoSolutions
Innovative, robust and cost-effective solutions
leveraging best-of-breed Open Source products.
Welcome
Open Source Projects
GeoTools
GeoServer
ImageIO-Ext
JAI-Ext
JAI Tools
OSGeo Foundation
Charter Member
GeoServer PSC
GeoTools PSC

Jody Garnett
Community Lead
jgarnett@boundlessgeo.com
@jodygarnett
Boundless
Provides geospatial tools and services for
managing data and building applications.
Welcome
GeoTools
GeoServer
uDig
OSGeo Foundation
Board Member
OSGeo Incubation Chair
GeoTools Project Officer
Eclipse Foundation
LocationTech Steering Committee
LocationTech Technology Project

GeoTools
GeoServer
GeoWebCache
OSGeo Foundation
GeoServer PSC
GeoWebCache Lead
Kevin Smith
Software Engineer
ksmith@boundlessgeo.com
smithkm
Boundless
Provides geospatial tools and services for
managing data and building applications.
Welcome

GeoServer at a Glance
Java Web Application to share and
edit geospatial data.
Publish data from any major spatial
data source using open standards.
Core Protocols
WMS – maps
WFS – vector
WFS-T – editing
WCS – coverage
WPS* – process
CSW* – search
Tile Protocols
WMTS – tiles
TMS – tiles
WMS-C – tiles

28 → 37 members of team geoserver (to be
updated
624 → 619 No clue this year (check with Kevin)
500+ → 525 pull requests
New contributors being added
(others stepping back)
Smaller base of active contributors this year.
Pull requests still going strong, very active
codebase.
Simone replaces Andrea as OSGeo Officer.
13 releases this year
GeoServer Project Status
How about team user-list?
2005 → 1400 (SF now checking on people
not active on list)
Also geoserver-italia, espanol and opengeo.cn
mailing lists

Release Schedule
stable maintenance
July 2.12.5
June
May 2.12.4
April
March 2.12.3
February 2.11.5
January 2.12.2
December 2.11.4
November 2.12.1
October 2.11.3
September 2.12.0
August 2.10.5 2.11.2

User List Participation
● Answering users questions relies on a low number of people
Action:
● We are sending reminder on effective use of the user list on subscription and
at the bottom of each mail
● Community building activity - remind developers about the definition of
“supported”
(not only reviewing pull requests, also answering user questions, used to be
obvious, but it’s not obvious to devs hired into open source)

● Bug stomp introduced in 2016
○ Relying on volunteer time not working out
○ allowed developers to reserve time
● Moved to fixed schedule in 2017
○ Arranging day each month too random
● Last Friday of each month
○ Check blog post for tips on participation
Standing “last friday of the month” bug stomp

Held “sprint” to update from
Restlet to Spring MVC for our
REST configuration API.
Thanks to GeoSolutions for
hosting, to Boundless and Astun
for sending developers, and to all
the sponsors who helped fund
this important and badly needed,
but boring work.
REST API Migration
2.12Viareggio Code SprintCommunity

REST API
Migration
● Migration complete
● REST API documented
○ Available online now!
2.12Viareggio Code SprintCommunity

GeoPackage:
● Provides an sqlite database containing vector or raster data
○ Initial idea from spatialite project
○ Now an OGC Standard to replace shapefile and personal file geodatabase
● GeoServer module
○ GeoPackage is a supported plugin in GeoTools
○ We can now promote it to a GeoPackage extension
○ Needs some work, splitting into 4 GeoServer packages, store, wfs output format, wms output
format, wps output format
○ Bug fix - raster mosaic geopackage had y-axis inverted.
GeoPackage (community)
2.12
GeoSolutions
Boundless
Community
Boundless Suite

● GeoServer importer supports GeoJSON files
○ Allows GeoJSON to be imported into PostGIS database and published in one step
○ Based on unsupported module in GeoTools
● Wish to package as a community module for GeoServer
○ Allow GeoServer to directly publish GeoJSON files
○ Text based, non indexed format, only suitable for small files
GeoJSON store
2.12Ian TurtonCommunity

● GeoServer importer supports CSV files
○ Allows CSV to be imported into PostGIS database and published in one step
○ Based on unsupported module in GeoTools (used as a tutorial for programmers)
● Package as a community module for GeoServer
○ Allow GeoServer to directly publish CSV files
○ Text based, non indexed format, only suitable for small files
CSV store
2.12BoundlessCommunity

CSV input/output for WPS
● Allows the input and output of CSV for use in WPS
processes
● This will lead to GeoTools CSV Datastore being
upgraded to community (and hopefully to supported)
● Come see more at “Building a Table Joining Like
Service with Web Processing Services” Friday
Afternoon
2.12
Ian Turton
Astun
RnD

● Initial version in 2.11
● Several fixes in 2.12, e.g.
accuracy, performance,
dateline crossing handling
● Being tested against global
mosaics of Sentinel2 (each
image in a different UTM
zone), more
improvements/fixes incoming
Heterogeneous CRS image mosaics
2.11/2.12Boundless/GeoSolutionsRnD/DLR
UTM60 image footprints UTM1 image footprints
Dateline wrapped
image mosaic

● Reprojected images could have shown border effects
Better handling of reprojection boundaries
2.10/2.11GeoSolutionsCommunity

● Higher order interpolation did not
play well with nodata, causing
border effects:
○ Bilinear
○ Bicubic
● Requires jai-ext enabled
Improved no-data handling in interpolation
2.10/2.11GeoSolutionsCommunity
Before
After

● Respect scale/offset attributes
● Improved support for related dimensions (e.g. time/runtime in forecast),
automatically finding the best related value when just one dimension is
provided
NetCDF reader improvements
2.11
Niels/Devon/Ben/Daniele
Scitus/Boundless/Transient/GeoSolutions
Customer
Runtime

S3 GeoTIFF Community Module
Support for GeoTiffs hosted on Amazon S3
● Reader based off of the GeoTiff reader.
● Very basic caching of images from S3 based off of EhCache
● InputStream implementation from JAI for reading imagery from S3
2.12
Devon
Boundless
Boundless
s3://landsat-pds/L8/001/002/LC80010022016230LGN00/LC8001
0022016230LGN00_B1.TIF?useAnon=true&awsRegion=US_WEST_2

Style Page
2.10BoundlessCommunity
Refactored CSS Style
Page into the main
application.
Works with
SLD, CSS, YSLD,
Mapbox
Improved experience for
everybody!

New features for CSS (2.10)
● Rendering transformations supported (feature parity with SLD!)
● Rule nesting, helps with selector duplication and makes styles more readable
/* @title Levels */
* {
transform: ras:Contour(levels: 1100 1200 1300 1400 1500 1600 1700);
stroke: black;
z-index: 0;
}
/* @title Values */
[@scale < 3000] {
transform: ras:RasterAsPointCollection();
mark: symbol('square');
:mark {
size: 2;
fill: black;
}
label: [GRAY_INDEX];
label-anchor: 0 0.5;
label-offset: 5 0;
font-family: Arial;
font-fill: black;
z-index: 1;
}
Extract contours at the desired levels
When zoomed-in enough, extract point
values, display center with a mark, and
value on top of it
2.10Andrea AimeCommunity

New features for CSS (2.12)
● Code completion in the editor
● LessCSS compatible color functions
○ Darken, lighten, saturate, ….
● More readable scale denominator rules
○ E.g. @sd < 100k
● Impressive CSS->SLD translation speedup
○ 50 times faster on large CSS sheets
● Easier env variable integration
(in progress, not yet landed)
○ @varname or @varname(defaultValue)

YSLD extension module
● Style Layer Descriptor represented in
YAML
○ Uses indentation rather than XML
○ Easier to read
○ More compact
○ More flexible syntax
○ Contains variables for reusable code
○ Compatible with SLD
● REST API can convert formats
○ Preview your existing styles as YSLD
using REST API
○ View the SLD representation of YSLD
2.11Boundless
symbolizers:
- polygon:
stroke-color: 'blue'
stroke-width: 1
fill-color: '#7EB5D3'
- text:
label: ${name}
fill-color: 'black'
anchor: [0.5, 0.5]
x-maxDisplacement: 40
x-autoWrap: 70
Community

● JSON representation of styling
○ More limited than SLD/YSLD/CSS
● Works well with vector tiles
○ Cross platform styling (OpenLayers, MapBoxGL)
● Work in progress:
○ Currently a GeoTools community module
○ Should package and go to extension if we get time!
MapBox Style Community Module
2.12David VickBoundless Suite
MapBox Style in GeoServer and
OpenLayers

Labelling improvements
● Kerning on by default
(adjusts spacing between chars, e.g.
Check space between T and r in Trimble)
● Underline and Strikethrough
● Char and word spacing control
● Work in progress:
○ Labelling border of polygon
(instead of the centroid)
○ Filtering labels based on
polygon screen size in pixels
2.11
2.12
Andrea Aime
Nuno Oliveria
Community/Geoinfo/
OpenGeoGroep

Improved QGIS Compatibility
● Additional custom “Well Known” marks
● Support for SLD 1.1 External Marks (e.g. TTF font based)
● SVG Parameters 1.0 spec
<PointSymbolizer>
<Graphic>
<ExternalGraphic>
<OnlineResource xlink:type="simple" xlink:href="firestation.svg?fill=#FF0000" />
<Format>image/svg</Format>
</ExternalGraphic>
<size>128</size>
</Graphic>
</PointSymbolizer>

● Label export, most shared configurations exported
● Some in the works (word/char spacing, strikethrough, polygon border labelling)
● Some changes needed in GeoServer SE 1.1 parser too
QGIS 3.0 style export improvements
2.12
Andrea Aime
GeoSolutions
OpenGeoGroep

Vector Tiles Extension
Extension for publishing vector tiles
- Great solution for modern High
Definition Screens
- Use for GeoWebCache for
“MapBox without MapBox”
- Couple with MapBox style for
consistent styling
2.11Gabriel RoldanBoundless Suite

Better centroid / label placement in “vector” outputs
KML
● generate placemarks that are
always inside polygon
(for concave polygons)
Vector Tiles
● Pregenerate label points in
tiles
○ Reduce effort on client
2.12
Kevin Smith
Just DeOlivera
RnD
Label Labelvs

WMS sortBy vendor param
● Sort by recency, cloud cover, and so on
● Supported by vector data and image mosaic
2.12
Andrea Aime
GeoSolutions
Community
Sort by recencySort by cloud cover

ncWMS like extensions to GetMap (community)
● A new styling language designed for dynamic colormaps
● Applies to all rasters based on their statistics
● Use can control min/max, log scale, animation
● Part of OGC TestBed 12 activities
2.10GeoSolutionsOGC

“null” prefix bug finally solved
● Occasionally GeoServer could return WFS GML with “null” as the feature
prefix
● Solved for simple features
● Solved for complex features
on global services
2.10
Nuno Oliveira
GeoSolutions
IGEA/NLS

App-schema improvements
● Improvements to connection pool usage (only one connection used
per request now)
● Some improvements helping while using MongoDB as a source for
app-schema records
● Hale mapping support being extended to support MongoDB
2.12
Nuno Oliveira
GeoSolutions
NPRA

GeoJSON output for complex features
● Complex features not limited to GML anymore
2.10
Nuno Oliveira
GeoSolutions
NPRA
{
"type":"FeatureCollection",
"totalFeatures":"unknown",
"features":[
{
"type":"Feature",
"id":"gu.93",
"geometry":{
"type":"Polygon",
"coordinates":[
...
]
},
"properties":{
"description":"Olivine basalt, tuff, microgabbro, minor sedimentary rocks",
"occurrence":[
{
"shape":{
"type":"Polygon",
"coordinates":[
...
]
},
"name":"urn:x-test:GeologicUnit:16777549126931093",
"observationMethod":{
"CGI_TermValue":{
"value":"urn:ogc:def:nil:OGC:missing"
}
},
"positionalAccuracy":{
"CGI_TermValue":{
}
},
{
"shape":{
"type":"Polygon",
"coordinates":[
...
]
},
"name":"urn:x-test:GeologicUnit:16777549126931093",
"observationMethod":{
"CGI_TermValue":{
}
},
"positionalAccuracy":{
"CGI_TermValue":{
}
},
"specification":{
}
}
],
"name":"urn:x-test:GeologicUnit:16777549126931093"
}
// properties ends here
}

Vendor param for mosaics: cqlFilter and sortBy
● Sort by recency, cloud cover,
and so on
● Filter on the same
● Maps to image mosaic read
parameters
● Uses attributes in the mosaic
index
● Consistent with WMS now,
first see and then download
2.12
Andrea Aime
GeoSolutions
Community

NetCDF input/output improvements
2.12
Niels
Scitus
Community
● Can configure NCML files as sources
● Support variable's attributes definition in
NetCDF output
● Preserve scalars from input
(NetCDF in/NetCDF out)

● GeoServer supports “cascading”
remote WMTS services
○ GeoServer acts as a client, rather than
just rerouting each request
○ Same as WMS cascading
● WMTS cascade is a work in
progress
○ Should land in 2.12
WMTS cascading
2.12
Ian Turton/Emanuele Tajariol
GeoSolutions
Customer

Parameter filter cache drop improvements
● GeoWebCache can cache based on parameters
like style, time, or elevation.
○ Could not truncate (remove cached tiles) across all
parameter values, each must be known and truncated
separately.
○ Tiles could be orphaned if the allowed values changed.
○ GeoServer can truncate tiles when the data within them
changes, but only caught the default values for parameters
other than style
● Added operations for
○ Cross parameter truncate
○ Orphan purge
● Enabled cross parameter truncate on data update
GWC
1.12
GS
2.12
Kevin Smith
Boundless
MapStory

WMS/WMTS multidim domain discovery module
(community)
● Data with N dimensions (time, elevation, custom ones)
● Dimensions can be related (forecasts, time and run time), the data can be
scattered and found in pockets, hard to locate data in the ND cube
● Add extra calls to query the domains
● Developed as part of OGC Testbed 12
● See the engineering report and give us feedback:
2.10
Andrea/Nuno
GeoSolutions
OGC
http://docs.opengeospatial.org/per/16-042r1.html#WMTS_Multidimensional_GeoSolu
tions

Security Consideration - Disabled writing by default
● WFS-T allows editing using
Transaction requests
○ May not be what you expect?
● Default configuration
includes security restrictions
on WFS-T functionality
● Restricting editing of data to
the administrator account.
● Makes service read-only
(while still advertising wfs-t)
2.11Nuno OliveiraCommunity

● Great performance improvement!
● Test with 10000 layers
Faster startup times for large data dirs
2.11AndreaCommunity
Version COLD
(Startup time seconds)
HOT
(Startup time seconds)
2.10.1 428 39
2.10.1 + JDCBConfig 62 49
GSIP 155 68 29

● Great performance improvement!
● Tested on topp:states to make catalog lookup times more evident (assume diff is a constant)
Faster runtime times for large data dirs
2.11AndreaCommunity
Version Throughput req/sec Avg resp. time ms
2.10.1 169 47
2.10.1 + JDCBConfig 68 117
GSIP 155 233 34

● When loading a shapefile the PRJ file only
matched on AUTHORITY code
● Faster lookup implementation allows
GeoServer to search the EPSG database
and match far more often
Better/Faster EPSG code lookups
2.11AndreaCommunity
This reduces a common frustration when configuring new layers in GeoServer

Module “status” REST API (and GUI)
● Verifying what modules are installed was a
challenge
● REST API access to the “status page”
● Easy to add additional entries
● Used to check environment for difficult to
configure modules that use native code
2.11
Morgan Thompson
Boundless
Boundless Suite
geoserver/rest/about/status

Parametric configuration
● Stick parameters in store configurations and
some security bits
● Helps migrating a data dir between
environments (e.g.,
integration/test/beta/production)
● Set the
-DALLOW_ENV_PARAMETRIZATION=true
system var
● Create a property file with variable values
2.10
Alessio Fabiani
GeoSolutions
Eumetsat
World Bank
store_url = file:///var/geoserver/store/teststore

Backup/restore (community)
● Save the configuration to a zip file (not the
data!)
● Restore later, in the same or different
environment
● UI to run backup and restore
● Asynch operations
○ During backup the configuration is forced read only
○ During restore it’s fully locked
2.10GeoSolutions
Eumetsat
World Bank

Web interface in Tools / Resource Browser
● Manage and edit resources
● Data Directory or JDBCStore
● Community module
● Packaged plugin coming soon
● REST API sporting equivalent functionality
found in core
Resource Browser (community)
2.10
Scitus
Development
Boundless/NGA

OAuth (community)
● Allows GeoServer authentication to be
provided by OAuth
● Allows users to login with:
○ GeoNode
○ Google
○ Github
○ … add yours here, it’s pluggable
● Developed for integration
with GeoNode
2.10
Alessio Fabiani
GeoSolutions
Boundless

Take care to report the issue in a responsible fashion:
● Keep exploit details out of issue report
● Mark the issue as a vulnerability.
● Be prepared to work with Project Steering Committee (PSC) on a solution
● Keep in mind PSC members are volunteers
and an extensive fix may require fundraising / resources
If you are not in position to communicate in public please consider commercial
support, contacting a PSC member, or reaching us via the Open Source
Geospatial Foundation at info@osgeo.org.
Security Vulnerabilities - Responsible Disclosure

Responsible disclosure policy is working:
● Most users contacted us in a responsible fashion
● One user shared confidential customer report on the user list
Where do reports come from?
● Majority reported from security audits
○ Usually performed under contract
○ Automated tools tend to focus on web administration ui, rather than web services
● GeoServer is subject to several security audits a year
Security Vulnerabilities

Upgrade to get security fixes
● Security fixes being added to supported versions only
● Please upgrade your GeoServer installations!

Java 9 Compatibility
Java 9 has been delayed until 2017, but we
would like to ensure GeoServer will function.
(Oracle stops free security updates to Java 8)
We have a couple incompatibilities with the
GeoTools “service provider interface” plugin
system that we will look at resolving during the
code sprint!

The Java Advanced Imaging library enables
GeoServer to do all kinds of great image
processing! The JAI-EXT project extends this
library with replacement no-data / footprint
aware operations.
Sadly Oracle has not maintained the project,
and has not released it as open source.
(see Replace-JAI on GeoTools wiki)
Raster Processing Engine
Joint OSGeo / LocationTech game plan:
● Raster Processing Engine API
definition of raster-processing-api
● Raster Processing Engine
reference pure-java implementation
● Raster Processing Wrapper
benchmark jai-wrapper implementation
● GeoTools Replace JAI
migrate to raster processing engine api
and coordinate transition affected projects
(GeoServer, GeoNode, uDig, GeoScript,
etc...)

OpenSearch for Earth Observations
● Recent addition to OGC protocol for search of earth imagery
● Simpler alternative to CSW with ebRIM
● Two steps search
○ First find collection of similar products (by satellite/sensor)
○ Then search products in it (by other attributes, e.g., cloud cover, off nadir)
● Keeps database of searchable metadata
● Links to OGC services providing said data
● Allows downloading ISO/O&M metadata
2.12GeoSolutionsDLR/EVO-ODAS

● Result is RSS with human readable descriptions and
○ Links to OGC services providing said data
○ Links to download ISO/O&M metadata
○ Links to download the raw data
● Exposes one feature type per collection, allowing easy mosaic setup with
same filtering and ordering

http://cloudsdi.geo-solutions.it/geoserver/oseo/search?
parentId=SENTINEL2
&cloudCover=30]
● E.g., search all SENTINEL2- images with cloud cover less than 30%

OpenSearch for EO: admin REST API
● Create/Update/Delete the collections and its information
● Create/Update/Delete the product and its information
● Either one shot (throw a zip file with all the info) or granular (edit
searchables first, then thumbnail, then metadata, links, …)
Swagger documentation

State of GeoServer 2.12

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to State of GeoServer 2.12

Similar to State of GeoServer 2.12 (20)

More from GeoSolutions

More from GeoSolutions (14)

Recently uploaded

Recently uploaded (20)

State of GeoServer 2.12