22 Feb What’s Coming in the next Arc2Earth Sync Beta
The next version of Arc2Earth will have much tighter integration between the extension that runs inside ArcMap and the Arc2Earth Sync application that runs independently of ArcGIS. In fact, with the new toolbars and ArcCatalog integration, you probably won’t even open the main Arc2Earth Sync user interface at all, everything is accessible directly from ArcMap. The new version is still several weeks away (and will continue to be in beta) but it’s a huge improvement in working with services that exist outside of the ArcGIS ecosystem.
We’re attending the ESRI FedUC next week (we’re in our partner’s booth New Light Technologies), stop on by and we’ll give you the inside scoop. We’re also co-sponsoring the FedGeoDay event on Thursday, 2/28 where we’ll be talking about Arc2Earth Sync and TileMill Connect.
Here are some of the highlights for the next version:
- All new startup and login, you will not be required to have a Google Account anymore. In many cases, if you’ve already activated your serial number, Sync will just start up automatically
- New and Rebuilt Table Providers
- Google Maps Engine – Google has been pretty quiet about this service, I expect we’ll be hearing a lot more about it in the near future. With Sync, we aim to make all the data types (vector and raster) easily accessible to ArcGIS desktop users.
- MapBox – We’ve talked about MapBox in the past here and here, it’s an amazing service for building and publishing your maps. The MapBox team is also second to none.
- CartoDB – We’re rebuilding to CartoDB connector to take advantage of all the new features they released in V2 of their service.
- Fulcrum – An incredible Mobile Data Collection service
- ArcGIS Online for Organizations – Esri clearly has this covered but we’re building this provider for older desktops and interop with other services.
- DropBox and Google Drive – We’re experimenting with multi-user editing using these services. The new editing subsystem (outlined below) allows us to proxy any file (shapefile, PGDB, FGDB etc) that you are sharing, without holding locks. This prevents the file corruption that will inevitably occur if multiple editors attach to files directly in DropBox. This process shows a lot of promise and if it works out, we’ll be completely scrapping our Table Sharing infrastructure in favor of these simple file sharing services. A shared, multi-user File Geodatabase for project based work would be a huge win for many of our clients.
- All new editing subsystem (outlined below)
- A new Arc2Earth Sync root directory in ArcCatalog, super easy access to all of your services.
A New Approach to Editing
In the current version of Sync, tables are fully downloaded from their source and kept up to date via editing events or by merging a new full download at set intervals (hourly or daily). This works pretty well but misses edits that occur in the source database until the next full synchronization is run. It’s also not practical for very large tables (1 million features+ etc). A new approach was needed that gives us access to fresh data, works with very large databases, provides offline caching when needed and works with ArcGIS for Desktop Basic (e.g. ArcView) editor license.
In the new system, we’re using two different methods for displaying tables in ArcMap. Query Layers and a custom Plugin Datasource. Query Layers appear to be the fastest way to communicate with ArcGIS from our local Sync service and since they are native ArcGIS datasources, they behave very well within the ArcGIS ecosystem. We’ve re-provisioned some of the work we did for TileMill Connect to make any external service look like a Postgres database running locally. ArcMap thinks its talking to PostGIS. The custom plugin datasource works well for older versions of ArcGIS.
In both case, ArcMap will treat these datasources as read only. To allow editing from the ArcMap Editor (or via scripts, code etc), we maintain a write-behind cache for each table. It has several options on how it is kept fresh but for large tables, it’s on-demand during map refreshes. As you pan the map, you’re seeing the results first while Sync is updating the cache in the background. Just prior to editing, the writable featureclass in the cache (a local File Geodatabase) is swapped into place in ArcMap. The good part is this happens automatically in the background, as far as the user is concerned, it’s the same layer in ArcMap. Featureclass extensions catch the edits and post them to their respective services.
It would be incredibly awesome if Esri made Plugin Datasources first class citizens in the ArcMap Editor but as of right now, they do not seem to work. If full read/write access comes to Plugins, we’ll definitely be using them but I would not expect it to be back-ported to previous version of ArcGIS anyway (10.0 and less).
We’re excited about this new form of display and editing in Sync, it’s a very simple and intuitive way to work with data services outside of the ArcGIS ecosystem.
The screens below show a Google Maps Engine table being displayed in ArcMap as a Postgres Query Layer