15 Nov Open Mxd Documents in MapBox TileMill
Update – Added a quick video at bottom showing TileMill Connect in action
In the current version of Arc2Earth Sync, we support building tile caches from your data and then automatically publishing them to MapBox for hosting. The tiles are drawn using the ArcGIS rendering engine and will match your existing Mxd maps symbols, renderers and labels exactly. We also offer some pseudo anti-aliasing to produce tiles that are visually more appealing than the default ones. For many users, this is all that is needed.
However, MapBox also offers a robust client side tile creation app called TileMill. It uses the Mapnik rendering engine and very intuitive CSS-like syntax called Carto to build and style maps. It contains a lot of functionality that is not available in ArcGIS, like native anti-alias drawing, Adobe photoshop-like compositing operations and easy to use zoom syntax for working with the standard tile cache levels (ex:
#my_layer [zoom > 15] ). For example, check out this blog post to see what’s possible.
It would be great to use your existing Mxd documents directly in TileMill but they both structure maps/layers/symbols in fairly different ways. Likewise, TileMill is not able to view all of the vector feature class data that you can display in ArcGIS. What is needed is twofold:
- A converter to dissect the contents of an Mxd document and translate the structure (layers, renderers, labeling etc) into the equivalent Carto and TileMill project format.
- Once loaded, a way for feature data locked in ArcGIS formats to display correctly in TileMill. A one time export to shapefiles could work but is not necessarily optimal for users who want ongoing access or have very large datasets. In it’s current form, this limits TileMill to shapefiles, SQLlite databases and PostGIS datasources. The traditional ArcGIS user has data in shapefiles, Personal and File GeoDatabases, SDE (version and non-versioned) and any number of other esri specific formats.
Enter TileMill Connect. It accomplishes the first item as best as possible and then uses a pseudo PostGIS proxy service (running and serving locally) to stream data from your Mxd’s into TileMill. Connect runs as an optional service inside of Arc2Earth Sync, you’ll need to turn it on before you start playing with it.
Mxd to Carto
Converting an Mxd layer hierarchy into it’s Carto equivalent is never going to result in a one for one match, the structures are just too different. However, we can extract enough common information to make the process worthwhile. The reality is there are a lot of Mxd files out there and manually recreating them is not a feasible option. Here’s what is converted:
- All Feature Layers are added to TileMill project. Shapefiles are added as native connections and all others are routed through the PostGIS proxy. Layer’s with Minimum and Maximum scale dependencies set will have the proper Carto zoom selector(s) added
- Simple, Unique Value, Class Break and Proportional feature renderers are converted into their equivalent Carto filters.
- Nearly all Marker, Line and Fill symbols are converted into Carto. Multi Layer symbols are added as Carto attachments or in the case of certain marker symbols, converted to SVG. Picture and Marker lines and fills always use flattened SVG versions of the markers. There’s also a Style Editor window where you can select or build any type of symbol and then immediately see it’s Carto equivalent. (and with an easy copy/paste, have it running in TileMill in seconds!)
- Most major label attributes are converted however this is work in progress. There is a lot of great label information stored in Maplex that we need to find a way to get out. Also, the flexible PostGIS proxy design allows us to server annotation layers as point tables, so all of your exactly positioned labels should translate ok.
Pseudo PostGIS Proxy
An original design for Connect relied completely on OGR’s ability to open all of the ArcGIS datasources and I’m certain that in the future, this will be a viable option. However, in the short term there are some issues with how well the drivers could deal with all “flavors” of ArcGIS data, TileMill not shipping any of the esri compatible drivers and most importantly, how we could deal with data that doesn’t even have a driver (TINs, Terrains, Lidar etc).
To get around these issues, we decided on a local proxy that would load Mxd documents into memory and then make the feature layers inside them appear to be PostGIS tables. It mimics the Postgres Backend wire format and handles just enough message types to make it appear as a read-only database table to TileMill (note, it is *not* a real PostGIS service)
To wire this up in TileMill, the connection string looks like this:
dbname=C:UsersPublicDocumentsArc2EarthLocalGovDemo.mxd host=localhost port=54321 user=sync password=sync
and to connect to individual table, you use standard SQL schema/table syntax:
All of this is handled automatically during the export from the Mxd into a TileMill project but you could add the tables manually from any Mxd that is accessible from the computer.
TileMill Connect is not in the current Arc2Earth Sync beta yet, we’re expecting it to be ready in early December. Based on user feedback, we’ll decide what to work on next. Serving raster data via the pg_raster syntax in PostGIS is an obvious choice. We’re excited to see what maps users can make when leveraging the power of both ArcGIS and TileMill together.