GeoServer and the Java toolkit GeoTools comprise one of OSGeo’s fastest moving mapping ecosystems. In addition to this core we will cover java-stack developments in JTS, GeoWebCache and ImageIO-Ext.
First up is “State of GeoServer” reviewing the new and noteworthy features introduced in the past year. Our six month release cycle sees GeoServer 2.5 and 2.6 being released this year. These releases bring together exciting new features: WCS 2.0 and WCS 2.0 Earth Observation profile, batch importer, and a fresh implementation of GetFeatureInfo.
Switching to technical for a GeoTools update. A passion for performance sees improvements in PNG and JPEG encoding, rendering from PostGIS and experimental JDK work. This presentation provides a review of new features, api changes and community modules. We have simplified the core FeatureCollection interface, introduced partial 3D support, structured grid coverages, multiple grid coverages and extended JTS Geometry with Curves.
Data formats support continues to grow with the latest GeoPackage, native NetCDF support and a new shapefile and wfs client implementation. Database users have table and index management methods to look forward to. Extensions have seen a lot of activity with WMS client improvements, vector grids, and a transform extension for dynamic feature collection processing.
Attend this talk for a cheerful update on what is happening with these great OSGeo projects! Whether you are an expert user, a developer, or simply curious what these projects can do for you, this talk is for you.
2. About your Presenters Contributors,
Guests, and Cameos
Andrea Aime (GeoSolutions)
Martin Davis (JTS)
Justin Deoliveira (Boundless)
Jody Garnett (Boundless)
Kevin Smith (GeoWebCache)
3. Outline
● Going to go fast - and have time for Q&A
● Intro
o GeoServer 2014
o GeoTools 2014
o JTS 2014 (Martin Davis)
o GWC 2014
o GeoScript 2014
o Friends
● Features
● Roadmap Wishlist
● Q&A
4. Money - Here is how this works
● Thanks to customers/employers/sponsors
we can share a preview of some work
scheduled for 2014
o We thank customers early and often
o it is how open source software gets made
5. Love - here is how this works
● If you have an idea or can help out
volunteers are always needed
o Contact details provided for active development
o Some wishes are ten years old
● Do not assume
o Anyone will pay for a visual SLD editor (sorry users)
o A volunteer is testing Java 8 (sorry developers)
9. GeoServer Highlights for 2014
● GeoServer 2.4: released September 2014
o Served as stable branch for six months
o Extended for six months as maintenance branch
● GeoServer 2.5: released April 2014
o Current stable branch
o Scheduled six months of maintenance updates
● Governance
o Transfer of codebase from OpenPlans to OSGeo
o New “community” modules → Graduated extensions
10. GeoServer: Health Checkup
● 28 members of team geoserver
o 779 geoserver-devel!
o Core group of very active contributors
o New contributors being added
● How about team user-list?
o 2290 geoserver-users
o 169 geoserver italia
o ??? GeoServer en Espanol
o ??? OpenGeo.cn (GeoServer and more)
o Some help with 2.5-beta … but many wait for 2.5.2
● Active well managed codebase
o 200 pull requests this year
o New! CONTRIBUTING.md file for GitHub
13. GeoServer moves to OSGeo
● Open plans is stepping
back from software development
● Strong relationship with OSGeo
o Vendor Neutral custodian to encourage participation
from additional parties
o Lazy: Participants had often already signed
OSGeo Contributors License for GeoTools
o See: GSIP-118
15. GeoTools
GeoTools is an open
source Java library
that provides tools for
geospatial data.
16. GeoTools: History and Organisation
● GeoTools 1: 1996-2001
o Leeds University
● GeoTools 2: 2002-Present
o OpenPlans, Refractions, GeoSolutions, etc ...
● Governance
o OSGeo Incubation 2006
o Graduated 2008
o Copyright OSGeo as
a vendor neutral organisation
17. GeoTools Highlights for 2014
● GeoTools 10: released September 2014
o Served as stable branch for six months
o Extended for six months as maintenance branch
● GeoTools 11.5: released April 2014
o Current stable branch
o Scheduled six months of maintenance updates
● Governance
o New “Unsupported” plugins → Graduated plugins
o New Committers
18. GeoTools Health Checkup
● 27 members of team geotools
o Active in bursts near release
o New contributors being added
● Active well managed codebase
o 203 pull requests so far this year
o New! CONTRIBUTING.md file for GitHub
23. GeoWebCache 2014
● Tile Cache
o WMS-C
o TMS
o WMTS
● News
o OpenPlans →
OSGeo
o Version 1.5 (2013 Q3)
o Version 1.6 (2014 Q3)
Kevin Smith
Junior Software Developer
Boundless Spatial Inc.
24. GWC: Mass Truncation via Rest
External management of GeoWebCache!
POST http://localhost:8080/geowebcache/rest/masstruncate
<truncateLayer>
<layerName>trekConventionDensity</layerName>
</truncateLayer>
Boundless Kevin Smith 1.5.0
25. GWC Features / Fixes
● GeoWebCache 1.5.0: August 2014
o Extensible Parameter Filters
o First release synchronized with GeoServer
● Fixes/Improvements added to 1.5.3
o WMS Metadata Passthrough
o WMS GetLegend Passthrough
Volunteer Eric Dahl 1.5.1
GeoSolutions Andrea Aime 1.5.1
26. GeoWebCache Roadmap (1.6.0)
● Read ArcGIS Compact Caches
o Generate with ArcGIS and Serve with GWC
● Improvements to Tile Fuser
Independent Bjoern Saxe 1.6.0
GeoSolutions Nicola 1.6.0
28. Spatial capabilities for
dynamic scripting
languages
● Groovy
● Scala
● JavaScript
● Python
Justin Deolivera
GeoScript
29. GeoScript: Update
● Jared working hard
o Curved geometries, GeoPackage, MBTiles
● Working on a common versioning scheme
● Standardizing on GeoTools version
● Early stages of incubation
● TODO: Logo :)
32. ImageIO-Ext (more formats)
Extensions, fixes and
for Java ImageIO:
● Support for GDAL
● JPEG2000 via
Kakadu
● Improved TIFF
● Now on GitHub!
Simone Giannecchini
33. JAI-Ext (more image processing)
● more features to
existing operations
● improved
performances
● developing new
operations
Nicola Lagomarsini
34. JAI Tools
Image Processing
● Tools for analysis
● Zonal-Stats
● Contours, vectorize
Michael Bedward
35. Jiffle
Raster Calculator
● Domain specific
language for image
processing
● C-Syntax for that
curly brace comfort
Michael Bedward
36. Features and Fixes
The magic of Open Source
Customer Implementor Release
37. WCS 2.0
A WCS version for human beings
DLR
Andrea Aime
Simone Giannecchini
Daniele Romagnoli
2.6.0
38. WCS protocol support
● New GeoTools classes to parse/encode
the XML documents in the WCS 2.0
protocol
39. Full GeoServer implementation
● Implementing the full protocol
● Including the GeoTiff
Extensions, controlling details
of the generated GeoTiff file
● NetCDF output (see later)
40. Did I mention the protocol looks sane?
http://localhost:8080/geoserver/wcs?request=GetCoverage&service=WCS&ver
sion=2.0.1&coverageId=geosolutions__NO2
&Format=NetCDF
&subset=http://www.opengis.net/def/axis/OGC/0/Long(5,20)
&subset=http://www.opengis.net/def/axis/OGC/0/Lat(40,50)
&subset=http://www.opengis.net/def/axis/OGC/0/elevation(300,1250)
&subset=http://www.opengis.net/def/axis/OGC/0/time
("2013-03-01T10:00:00.000Z","2013-03-01T22:00:00.000Z")
42. Many coverages per reader
● A GeoTools reader can read and
advertise metadata for multiple coverages
now
● GeoServer updated to follow suit
43. StructuredCoverageGridReader
● Drill down in the various dimensions
● Interact with slices, list, filter, add, update,
delete (GranuleSource/GranuleStore)
● Harvest
entire
new files into the
sources (mosaic
only)
44. GeoServer support
● Publish NetCDF/Grib as any other format
● Configure WMS-T dimensions
● Return a multidimensional array via WCS
2.0 NetCDF output format
http://localhost:8080/geoserver/wcs?request=GetCoverage
&service=WCS&version=2.0.1&coverageId=geosolutions__NO2
&Format=NetCDF
&subset=http://www.opengis.net/def/axis/OGC/0/Long(5,20)
&subset=http://www.opengis.net/def/axis/OGC/0/Lat(40,50)
&subset=http://www.opengis.net/def/axis/OGC/0/elevation(300,1250)
&subset=http://www.opengis.net/def/axis/OGC/0/time
("2013-03-01T10:00:00.000Z","2013-03-01T22:00:00.000Z")
52. GetFeatureInfo issues
● Most software does a small square bbox
query
● Symbol sizes took into account to an extent
(issues with dynamically sized symbols)
● No care for symbol shape
● Solution → actually draw stuff and check
pixels
53. Dynamic sizes
● Size proportional to
attribute
● Before, either miss
the big circle, or
report the small
one as being hit
● Now right match
54. Shapes and offsets
● The new code
won’t report a hit if
you did not click
the symbol
● Takes care of odd
symbol shapes,
and offsets
56. Represent curves in memory
● Circular arc by 3 points
● CircularString: sequence of the above
● CompoundString: mix straight and circular
arcs
● Use in polygons as well
57. Smart linearization
● When rendering, linearize
according to the rendering
resolution
● Otherwise, if necessary, do
so with a given tolerance
(e.g., when using JTS
topological operations)
● Selected code paths just
keep the curves
66. Java rasterizer scalability issues
● Rasterizing: turning vector data into
antialiased images (maps)
● Basically two choices
o Use Oracle JDK (Ductus), binary, closed source,
fast rasterization, but scales up poorly
o Use OpenJDK (Pisces), pure java, open source,
slower rasterization, but scales up well
67. Marlin rasterizer
● Fork of OpenJDK Pisces
● As fast as Ductus
o as scalable as Pisces
o Testing a complex
multiscale map
with 10 concurrent
clients at different
zoom levels
68. Marlin Easy Install
For Windows/Linux, Oracle or OpenJDK:
1. Download one jar from
https://github.com/bourgesl/marlin-renderer
2. Add these options to the JVM-Xbootclasspath/
a:/path/to/marlin-X.Y.jar -
Dsun.java2d.renderer=org.marlin.pisces.PiscesRenderingEngine
3. Done!
69. Marlin in Java 9?
● The author, Laurent Bourges,
is trying to get Marlin merged in Java 9
● Lack of response from Oracle
● Please Oracle, we care about
performance… don’t you?
71. Filter Interfaces - Technical Debt
GeoTools 2.3 introduced Filter 1.1 Interfaces
● Budget only covered new implementation
o GeoTools tests still used classes directly
o GeoServer and other apps also
● Two options for Filter confuses developers
o Class or Interface/Factory?
o What do I import to make this code work?
OpenGeo Justin Deolivera 2.3
72. Filter Cleanup - Cleanup
GeoTools 12.0 Cleanup
● Cleaned up with / careful refactoring!
● Test for XML Parsers / WFS were hardest
GeoServer 2.6.0 Cleanup
● Cleanup went smoothly
● BBOX Filters caused trouble
Boundless Jody Garnett 12.0
74. WFS “Next Generation” client
Ground up implementation of wfs-client
● Using GTXML parser (yay!)
● Based on ContentDataStore
(implementation is much easier to follow)
● WFS-T 1.1 ← exciting!
75. WFS Cascade
● GeoServer surfaces the wfs-ng as the ability
to “cascade” content from remote (or
internal) WFS Servers
● Performing 2.6 preflight testing wfs-ng was a
marked improvement over the origional
o We have cut over “cold turkey”
o We asked the geoserver-users list for help testing all
those tricky INSPIRE services
● wfs-ng includes configuration overrides
o You can account for several common mistakes
(such as servers that get confused about axis order)
77. gt-render-wkt merged to gt-render
● Formally an optional unsupported plugin
o This was a hidden “power tool” for great maps
o Create symbols using WKT directly inside your SLD
<WellKnownName>
wkt://LINESTRING(0.0 0.25, 0.25 0.25, 0.5 0.75, 0.75 0.25, 1.00 0.25)
</WellKnownName>
● Merge with gt-render
o Available by default for everyone
o Go out and make maps!
79. New mark factory for windbarbs
● Use WMO compliant wind barbs
● Well known name
template:windbarbs://$(value)[m/s]?emisph
ere=(n/s)
● E.g.: windbarbs://25[m/s]?emisphere=n
81. ResourceStore (Internal)
● GeoServer DataDirectory
o Default is “in the web apps folder”
o Copy to separate location when upgrading
o Can be split up several directories ← why?
● Optional to Store in JDBC Database
o “Catalog” configuration stored in Database
o “Data Directory” still used for SLD, Icons, Fonts
● ResourceStore is “Step 1” of fixing this
o Use “Resource” rather than “File”
Direct access to JDBC BLOB
“check out” file from the Database (if needed)
Boundless Jody Garnett 2.6.0
82. New GeoServer Website
One stop shop for everything GeoServer
Boundless
Justin Deolivera
Jody Garnett
2.6.0
89. SOLR data store
● Connect, display, filter data stored in your
SOLR server
● Expose all of them to the web via
WMS/WFS/WPS
NSW-GS-AU GeoSolutions 2.7.x/2.6.x
90. Clustering asynch WPS
● The only OGC operation maintaining state
● Improve GS so that the state gets shared
across cluster nodes
Load balancer
WPS1 WPS2 WPS3
Shared async
state
GeoSolutions GeoSolutions 2.7.x
92. GeoFence community module
● Advanced security rules in GeoServer
● Separate web app for config, GS plugin to
drive security
● Fully open source, already in use by some
GS based projects (GeoOrchestra)
GeoServer GeoFenc
e plugin
GeoFence
UI and REST
GeoSolutions GeoSolutions 2.6.x
93. CSS module for GeoTools
● Port of the GeoScript
Scala version to Java
● Allow more people to
contribute
● Faster translation to SLD
● Adding LessCSS features
down the road
/* @title Population < 2M
*/[PERSONS < 2000000] {
fill: #4DFF4D;
fill-opacity: 0.7;
}/* @title 2M < Population < 4M
*/[PERSONS > 2000000] [PERSONS
< 4000000] {
fill: #FF4D4D;
fill-opacity: 0.7;
}/* @title Population > 4M
*/[PERSONS > 4000000] {
fill: #4D4DFF;
fill-opacity: 0.7;
}
Andrea Aime Andrea Aime 13.x
95. Upgrade Wicket User Interface
● User interface is slowly falling apart
o No longer works with latest browser
(Current advice - use Firefox!)
o Need to upgrade to newer version of “Wicket”
● Why?
o User interface is not fun enough to attract volunteers
o Popularity of REST API
96. Update CITE Tests
● CITE Tests are the OGC conformance tests
● Pass them and you get a cool sticker for
your website
o Well only if you pay
o Idea: Ask OSGeo Board to talk to OGC?
● Tests keep us all safe!
o We would like some time to upgrade / fix
o Help out at the Code Sprint this weekend
97. GeoServer: WPS 2.0
Money/Love needed for:
● WPS 2.0 Operations
o Improved Job Control (stop process?)
● WPS has CITE tests!
98. High DPI Mapping
● OL3 and GeoServer can already show
wonderful images on High DPI displays
● To roll this concept out across the stack
o GWC to Tile based on “dpi” format option
o GeoTools rendering external graphics
99. GWC Cloud Storage
● Add options for cloud storage:
o Amazon S3, Acumulo, Azure, Google, OpenShift
● Mad ideas:
o Integrate S3 with AWS’s CDN
Boundless Kevin Smith 1.7.x
100. GWC Storage Filter
● Processes tiles before storing them
● Specific implementations can:
o Image Processing: color balance tiles
o Re-encode: Changed PNG z-compression level
o Change colour depth: PNG24 → PNG8
o Run external tools: PNGCrush etc ...
Boundless Kevin Smith 1.7.x
101. Geometry + Geometry + Geography
JTS Project has limited scope:
● GeoTools extends Geometry for Curves
● ISO 19107 Point/Curve/Surface
● Spatial4J looking at Geography
● GML Geometry wrappers around JTS
Money/Love needed: Geometry needed for
Open Source GIS industry
104. Q&A
● Q: When will be Java port of
CSS Style be available? Interested in using
it from MapFish.A: Andrea - It is a
background activity
need several more months on it.
● Q: Coudl not use as our State Plane defaults
to meters and we measure in feet? I
reported a bug last monthA: Feet? That is a
terrible idea! Still hard to get answer this
close to foss4g - will you be at the code
sprint? Thanks for bug report.
105. Q&A
● Q: Would anyone here be interested in a
GeoTools Cookbook?A: Please contact Ian
Turton
Editor's Notes
Go through related projects and see how we are running:
- JTS
- GWC
- ImageIO-Ext
- JAI-Ext
- JAI-Tools
- JAI-Tools Jiffle
- others?
This is just to give a project name, logo, context and contact.
Save for the Feature and Fixes later!
License and Governance
Proposed as LocationTech project
Eclipse Distribution License (BSD!), Eclipse Public License
Releases
Current - Version 1.13 - 2012
Next - Version 1.14 - 2014 Q4
Go through related projects and see how we are running:
- JTS
- GWC
- ImageIO-Ext
- JAI-Ext
- JAI-Tools
- JAI-Tools Jiffle
- others?
This is just to give a project name, logo, context and contact.
Save for the Feature and Fixes later!
Reminder - truncation is clearing the cache
Go through related projects and see how we are running:
- JTS
- GWC
- ImageIO-Ext
- JAI-Ext
- JAI-Tools
- JAI-Tools Jiffle
- others?
This is just to give a project name, logo, context and contact.
Save for the Feature and Fixes later!
Topics:
- Marlin
- PNG Compress (Boundless/Ordnance Survey)
- PNG Encoding Performance (GeoSolutions)
- Rewriting JAI Operators to be footprint aware
General Approach (and this will take up the bulk of the presentation)
1) Separator Slide giving the issue a name and Contact
2) Showcase the feature (or issue), explaining the context
3) Thank customer and team doing the work
4) Any good stories about the fix
5) Where the fix can be found / applied / used
6) Any next steps for funding or volunteers