What you need to get started:

• ArcView 9.2 or greater
• Arc2Earth Sync Beta (signup here for free)
• Google Account (signup for a Gmail account here)
• ArcGIS.com Account (signup here for free)

Signup for the Sync Beta and Login to Sync

To get started, signup for the Arc2Earth Sync beta and authorize your account so you use Google Fusion Tables as the backend database for your data. Once you’ve authorized your account, startup Sync and login using your Google and ArcGIS.com accounts.

Create a Table

Open the main Table Explorer window and click on the Add Table button. The wizard will walk yout hrough the process of creating a table manually or uploading an existing one to your Table Provider (Fusion Tables and CartoDB in the beta, more to come soon)

For this example, we’ll create a polygon table in Fusion Tables with two string fields. For the second field, we want easy and controlled data entry so we’ll setup a Coded Value domain that will be honored by all the ESRI apps (web, mobile and ArcMap). We’ve streamedlined this process so all you need to do is enter a comma delimited list of values for the domain. Click the Create button wait a few seconds while the table is created in the local cache and on the Cloud service.

Once the table is created, we need to publish it to ArcGIS.com so its available to all of the mobile clients automatically. Open the table’s Properties window and goto the Publish tab. Set the ArcGIS.com slider to “On” and click Ok to save the options. While saving, Sync is publishing an ESRI web map to your ArcGIS.com account, into a special folder called Arc2Earth Sync Maps. You’ll now be able to interact with this table on any of the mobile apps (Android, iOS, Windows etc) simply by going to that folder. To share this map with other ArcGIS.com web and mobile users, add it to a Group or share it with individuals.

Open in ArcMap

Select the table in the listview and click the “Open in ArcMap” button on the toolbar, this will either add the layer to an existing ArcMap or open a new one. You now have a live connection with Fusion Tables and ArcGIS.com, as edits come in from mobile apps, you will see them automatically in ArcMap. Likewise, as you edit in ArcMap, they will be streamed automatically up to the cloud and will be visible to all of the mobile clients (note – some of the ArcGIS mobile apps cache data so a map refresh might be necessary to see new data). Also check out this blog post on Table Sharing to see how could further the reach of this table by sharing it with other Sync users located anywhere in the world.

Add Data from the Web or Mobile Client

Now let’s make some edits from three different clients (ArcMap, ArcGIS.com web app and the ArcGIS Android app) and watch as they automatically synchronize between your local cache and the backend Google Fusion Table

Photos and Attachments

A great feature of the new mobile apps is the ability to attach photos and documents to any feature you’ve editing in the field. The was previously a tedious process that required matching up feature Ids and photos timestamps once data was brought back to the office. Now it happens automatically and even better, Sync will download any photo attached to a feature into a folder on your local computer. You’ll also be able to view it online using your Sync admin pages or as thumbnails images in your popup Info Windows in other apps

Online Feature Page screen grab

It was this same type of frictionless data sharing that we wanted to bring to ArcGIS users looking to work on small, ad-hoc projects with a list of remote users, some of whom were not always online. For early Arc2Earth users, this was accomplished using KML files that were edited by remote users in Google Earth and then emailed and imported into the master file back at some home base. This actually worked once it was setup correctly but has a myriad of small issues that when taken as a whole, make the solution sub optimal at best.

Table Shares in Sync look to change this completely. A Share is time based access to a private table in any Sync Table Provider (Google Fusion Tables, EarthBuilder, CartoDB etc) that *just works* from the Sync desktop client. There’s not much needed to share a table, just the email address of the intended user. An notification is generated and sent automatically, they would only need to click on the link to accept the share. After that, their Sync client will add the table and immediately start downloading the latest data. If you set the Share to expire at a certain time, the table automatically disappears from their table list. All edits through the share are done via a proxy so those users never have native access to the underlying table (say, in Fusion Tables) so you can be assured that Sync is the only way they can access your data. Once the share is in place, they can interact with the table in ArcMap/ArcCatalog/models as they would with any other local geodatabase. If they went offline, they could continue making edits in ArcMap that would subsequently be synchronized when network access was restored.

Another benefit of DropBox shares is the folder is available online from the website. With Sync, this applies as well except that access is available using standard APIs instead of by download. So, if editing data via ArcMap on a laptop in the field is not available, they could use the ArcGIS iOS or Android app to collect data instead.

To share a table, open it’s main properties window and goto the Shares tab

This would generate a invite email that looks like this:

After accepting by clicking on the link, the table would immediatley appear in their Sync client. Start editing!

The best way to understand Sync Table Shares is to try it out, you can sign up for the beta here. Invite a friend or two and share a table between the group, you can all make and see edits in near real time.

First, we overhauled how Table Providers are used by the desktop client. The original design had assumed a user would choose a provider (Fusion Tables, Earth Builder CartoDB etc) and use that exclusively for all of their work. However, what we found was users wanted to mix and match tables from different locations or accounts. So, in this version, there is still a default provider (Google Fusion Tables) that needs to be authorized on setup but there is a also a new “Connect” button that allows you to add any number of other providers as well.

Second, and the primary reason for this post, is the addition of the CartoDB provider in the beta. CartoDB is an excellent GIS cloud service, a lot like Fusion Tables except more power is exposed to the user. Under the hood, CartoDB is PostGIS, portions of TileMill and a lot of extra code to host your data and expose it via SQL and Map Tile APIs

By adding a CartoDB Provider to Sync, we make it available directly in ArcGIS. You can edit the points/lines/polygons in ArcMap and watch as it changes in the online CartoDb tables (and map tiles). We also allow it to work with other services like ArcGIS.com or any of it’s online APIs and applications.

One other item of note in this provider, its the first one to take advantage of Sync’s ability to translate ArcGIS renderers and symbols into other formats that can be used to draw maps outside of ArcGIS. In CartoDB’s case, that other format is Carto from DevelopmentSeed (the makers of MapBox/TileMill ). Carto is an excellent format for map symbology/labels and is drawn via the excellent Mapnik graphics library, so the results are visually beautiful.

In CartoDB, Carto is used as the custom format for styling and labeling any of your tables. Sync will translate the symbols used in any simple, uniquevalue or classbreaks renderer in ArcMap, into the appropriate css attributes. What’s better is that Carto supports “attachments”, or the ability to apply multi-pass rendering of different drawing attributes to the same feature on the map. The best example of this is drawing streets, one wide line is drawn as the outline while a second smaller stroke is applied over the top (after all of the features have been drawn). This makes for a smooth transition and visually appealing linework. ArcGIS has a very large set of symbols that use multiple levels to achieve the desired result, Sync will translate all of these into the appropriate “attachment” for the style. It will then upload the style when it detects a change, keeping your online map looking just the way you want it (note – this can be optionally turned off)

Here’s some sample multi-layer line symbols from the standard ArcGIS style gallery applied to a CartoDB based table. You can see the levels, colors, dash-arrays and x/y offsets are all applied correctly to the Carto CSS attributes. There is a limit to what can be translated but it seems to be the best correlation between ArcGIS symbols and another format that we’ve run into so far.

And here’s the live result up on cartoDB

There is currently a known issue on CartoDB that prevents url-based images to be used in a carto stylesheet, however, when this is fixed, all of your ArcGIS markers, marker lines and marker fills should work as expected. Many of our users have complained that Fusion Tables does not support custom markers for symbols (an issue Google is working on) so I’m hoping the CartoDB folks get this worked out asap

CartoDB is a great service with flexible pricing, give it a try for free and see if its a good fit.

Arc2Earth Sync is a new service that provides a bridge between your local ArcGIS data and the new, evolving world of Geospatial Cloud Services.

On the desktop, Sync runs as a small utility that synchronizes your local ArcGIS data with any number of online Table Providers. For the Beta, we support Google Fusion Tables and CartoDB however we have many more in development. You can share these tables with other ArcGIS users (9.2 or greater) to create cloud based, collaborative layers. All edits from ArcMap or mobile clients are automatically synchronized with the online provider and your local cache. Here’s a quick video showing the live editing process from ArcMap and mobile clients.

As a service, Sync provides a host of features that extend and homogenize the data from other providers. For example, Sync will make all of your Tables appear as OGC Rest and WFS layers for use in ArcGIS.com or other GIS software. It also provides full edit tracking and history for each of your tables as well as Edit Rules/Actions that can be applied before or after an edit takes place.

Arc2Earth Sync brings some peace of mind to your Cloud deployment. A local copy of your data is always current and available. You can work online or offline and be confident your data will always be correct. And switching between Table Providers for all or just a few tables is as easy as a button click. Try out a new service and switch back to the original provider if needed. Or mix and match tables from different providers, use the best features from all of them.

Table Providers in Beta

• Google Fusion Tables
• CartoDB (hosted PostGIS and TileMill service)

Table Providers in Development

• Google Earth Builder
• GISCloud
• GeoIQ/GeoCommons
• ArcGIS Online for Organizations

Find out more about Arc2Earth Sync here or signup for the Beta and try it out now.

We’ll be at the ESRI Federal GIS Conference tomorrow at booth 500 (New Light Technology), come say hi and we’ll show you how Sync can work for you.

I also participated in the GIS talk given by Google’s Mano Marks and Josh Livni. And by participate, I mean jam as much information into 10 minutes as possible while trying not to mess up some live demos. Here’s the link to the session. The room was about three quarters full which is much better then expected given the topic and timeslot.

My only regrets for the week were

• a) not going up to Sergei and saying hi (he was actively wandering the conference, mostly by himself without a huge entourage)
• and b) not trying the indoor stair-slide in the Google SF office

Arc2Earth Data Services

Data Services is a new product for use with Arc2Earth Desktop. It allows you to display Google Maps Street and Imagery data directly in ArcMap (legally, A2E is now an OEM of the Google Maps API). It also includes access to other Google Maps services like Geocoding, Routing and Elevation ( although these are still in beta for this initial release. our beta testers insisted the imagery service be released as soon as possible!) Future versions will also include some very cool Google Geo Cloud integration as well but you’ll have to wait for a future date to hear the specifics

Check out Getting Started with Google Imagery to see how easy it is to get started using this new service. Also, here’s a video showing how the imagery looks in ArcMap.

We have some new pricing and bundles available but one I’d like to highlight is the introductory offer for Arc2Earth Data Services and a free Arc2Earth Pro seat. You can’t beat this value, so get it while it’s hot (under the Professional Bundle section)

MapBox and MapStream Support

MapBox is a set of open source projects from DevelopmentSeed that allow you to create and host map tiles. TileStream is a paid service that gives you easy upload and management of your maps with a single monthy price. MapBox for iPad is a free application that allows you to download tile caches and bring them into the field without the need for an internet connection. Offline maps on the iPad are a compelling alternative to technologies like GeoPDF with the added benefit of exactly matching your existing desktop map symbology, labeling and renderers.

Arc2Earth now allows you to create map tile caches that are compatible with both MapBox for iPad and TileStream. You can easily export your maps directly from ArcMap and immediately have them available online or in the field. (also note that with this version, Arc2Earth Pro can create map tiles that can be used with MapBox)

For more information, check out this quick tutorial

Google IO 2011, their annual developer extravaganza, is right around the corner and Arc2Earth was lucky enough to be selected as a Developer Sandbox attendee. We’ll be in the Geo area of the sandbox both days, talking about all things Arc2Earth and Google. Come on by to talk shop or just say hi!

Additionally, I’ll be a guest speaker during the GIS with Google Earth and Google Maps session given by Josh Livni and Mano Marks (Wed, May 11th at 4:15). I’ll be talking about how Arc2Earth integrates with existing and future Google Geo technologies.

Typical Arc2Earth Cloud url on your own AppEngine Account: http://myInstance.appspot.com/a2e

With Arc2Cloud, you would use this instead: http://beta.arc2cloud.com/myInstance

Everything else about how you interact and build applications around the APIs is the same. The only major difference is the use of Google Fusion Tables as the datastore for all of your spatial and attribute data. If you plan on keeping your data public and have modest usage needs, Fusion Tables if free. If you need private data or much larger usage limits, you’ll need to get Premier access from Google.

Loading maps and data into Arc2Cloud is easy and for most users, it will take place directly from ArcMap. Simply select the layers in your map that you want to upload and click the upload button. In the background, Arc2Cloud will create Google Fusion Tables that exactly match your local layers (using OAuth to securely interact with Fusion Tables on your behalf). Once complete, an Arc2Cloud datasource that points at the newly created table is registered in your instance and finally, the data is batch uploaded to the table. By breaking this process up into batches, we can upload much larger quantities of data to FT.

Once uploaded, you’ll be able to interact with the Map via the built-in and external viewers. There will be a future post on the Google Earth viewer but as a preview, it contains full auto-complete search, editing and automatic KML regions for all of your data (no configuration, no static exports).

Your map is also available as a set of ESRI REST services as well (using the GeoServices REST spec). You can then use the excellent ESRI apps and APIs to build customer applicaitons or to tie your data into existing ESRI based ones. The maps are read/write (given proper credentials) so you can collect data online and your Fusion Tables will be automatically populated.

Another powerful feature is the ability to import Map schema and data from other online sources.

ArcGIS Server REST importer – Will import a MapServer or FeatureServer and create Fusion Tables for all layers. If it’s a FeatureServer, the renderer, labeling and feature template information is imported as well. (note – you should only import public services where there is no copyright infringement, we’ll likely build a bunch of free templates for different types of services)

Fusion Tables – Simply registers an existing Fusion Table (your own or one that is publically available). The table must have a Location field and while Arc2Cloud does not care about mixed geometry types, many other services that will try to interact with your table via Arc2Cloud will expect tables to have uniform geometries (e.g. all points or all polygons)

Twitter – This importer is not currently available (we are waiting on some cool new features in Google AppEngine to become available) however when ready, it will be a continuous importer. that is, you tell it a search query, an optional location and a time period and it will monitor the main twitter feed on your behalf, writing geo enabled tweets to a specifc Fusion Table.

There’s also the base API for Arc2Cloud, where you can create maps, datasources and features on the fly and FusionTables will be created in the background. This is great for ad-hoc applications where you don’t know what needs to be collected until the time comes.

It’s hard to believe what started out as an afternoon project to convert ArcGIS data into KML turned into something I’m still working on five years later. So first and foremost, I’d like to thank all of our users, ESRI for a great developer platform, Google for blowing the lid off geospatial and everyone else who’s supported this project over the years. The design right from the beginning was to keep it small and efficient, so during that time we’ve always been profitable and have never taken on outside investment and still somehow managed to sell to users in over 30 countries. Many thanks to our resellers for their help!

Arc2Earth is now a full featured KML exporter/importer, a map tile cache creator, a fast map tile cache consumer, a workflow tool, a geospatial Cloud provider, a data provider for Google’s Map API in ArcGIS and my current favorite, Arc2Cloud, a multi-tenant, read/write geo repository using Google Fusion Tables (1)

There’s been lots of ups and downs during this time and like most small companies, we’ve been down a few blind alleys. But in the end, I think it’s always made the product stronger and quite frankly, it’s been a great learning experience as well. By my read, the geospatial industry is currently in a period of change, innovation and the economic realities of cloud sourced services have the potential of really changing the current software procurement process. While I expect all current players to be around, I do think things will look a lot different 5 years from now. I’m confident we’ll be able to keep ahead of the curve but it won’t be easy, James is certainly correct in saying: The World… She be a Changing…

On another note, I’ll be doing a talk at this years ESRI DevSummit about some functionality in Arc2Cloud and given the brief time allotment, I’ll be doing a small brain dump on this blog as a primer to the talk itself. If you’re attending, please stop on by!

1. – I believe I’ve hit my max number of concurrent programming projects so if you’re in the market for a dev job, please drop me a line. I’ve got loads of new and interesting ideas lined up

We’ll be at the ESRI FedUC next week in NewLight Technologies booth if you want to stop by and chat about the new features, any issues you are having or just talk shop. 2011 is shaping up to be an interesting year in for geo technologies, we’ll try to stay ahead of the curve!

Fusion Tables Upload

Google Fusion Tables now support spatial queries and in combination with their amazing builtin mapping visualization, I expect them to be a very popular way to publish your data online. To help the average ArcGIS user work with this new service, we’ve built Fusion Table upload directly into Arc2Earth. You can still export your data to KML first and use Google’s web based uploader (or for that matter, this excellent shapefile uploader) however it’s much easier to use the builtin A2E tools since you can work directly against your FeatureLayers (which may be shapefiles, PGDBs, FGDB, SDE etc). It will also export the current renderer value for each feature and create additional columns to hold the symbology values.

Initially, you’ll need to manually set these values using the Fusion Tables web UI but as their API expands, we’ll be able to set the values automatically (and without the need for the extra fields). Here’s a quick video showing the process:

There’s additional Fusion Tables functionality built into Arc2Earth Data Services but that’s a blog post for another day.

Map Tile Caches

• 8bit PNG image format (yea I know, finally…)
• Command Line Builder – Click the menu directly off the Tile Exporter window and all of your current settings will be made into a valid command line. You can optionally launch the exporter to cut your tiles outside of ArcMap.
• Change Detection and Adaptive Exclusion Mask – There’s been some improvements to the Change Detection algorithm to deal with issues on 64bit servers. Also, a new Mask was added that works so well in certain situtations that it’s now on by default. The Adaptive Exclusion Mask will continually detect blank areas in your map as it exports levels, tiles below these areas in lower levels are then skipped when the exporter comes to them. If your maps contains a lot of transparent areas (like streams or discontinuous park polygons), this process can dramtically speed up exports. One samples export went from 20 minutes to 3 minutes with adaptive masking turned on.
• WeoGeo Cache Format – We’re working with our friends over at WeoGeo to add full tile cache export for use with their API. Initially, you can choose WeoGeo from the cache format dropdown to create tiles. In a later release, the tiles, preview images and necessary metadata will be packed up into a zip file, ready for publishing to WeoGeo’s growing list of online datsources

Arc2Earth Data Services (beta)

While there have been several plugins to view Google’s Street and Imagery data inside ArcMap, all of them rely on downloading Google map tiles directly. This is not supported by Google and is in violation of their TOS. We actually removed this functionality from our Map Tile Layer way back in Arc2Earth V2. That being said, many of our users have always asked for access to Google’s data as a both backdrop and to help position their own data in ArcMap

Well, we’re happy to say that with Arc2Earth Data Services, you now have access to thier worldwide imagery data for a yearly, per user fee. The Google Maps toolbar, which was previously only to be used by Premier/Enterprise users, can now be used by everyone who joins the program.

This service will also include the other Google Mapping APIs as well. So worldwide geocoding/reverse geocoding, elevation data and directions/routing. We’ll be adding simple toolbars to access these services directly from ArcMap

We’re still in beta now but if you want to help test this functioanlity or provide valuable feedback, please fill out this survey and we’ll get you on the list. Even Arc2Earth Community Edition users can participate!

Arc2Cloud (private beta)

Arc2Earth Cloud Services has been well received by our larger, more technically savvy users. However, Arc2Earth got it’s start by serving the everyday ArcGIS user and by helping them make the most of their limited budgets and time.

Arc2Cloud will now bring you one step closer, one click publishing of your maps to the Cloud. You can be up and running in minutes with your data available (read and write!) in Google Maps, Microsoft Bing (using map tiles), Polymaps, Google Earth (with automatic search and regions) and in ArcGIS.com (and all of its great applications and APIs).

Arc2Cloud has the exact same featureset as Arc2Earth Cloud Services except that all of your datasources are hosted in Google Fusion Tables. So that means your Fusion Table data will appear as GeoJson in OpenLayers and Polymaps, Regionated KML in Google Earth, as a WFS-T service or as an ArcGIS Server to the myriad of applications and APIs on ArcGIS.com. We’ve also added some great upload tools to Arc2Earth Desktop so you can get an entire map in Fusion Tables and registered in Arc2Cloud with a single button click!

Now you’ll get best of both worlds, Google’s amazing Maps and Search services and compatibility with the growing number of GIS and ArcGIS Servers in the world.

Since it’s read/write, you could immediately start using the ArcGIS iPhone application to collect features in the field. Your data will be synchronized with both Fusion Tables and your local desktop.

I’ll be attending the ESRI DevSummit in March to (hopefully) talk about geoprocessing with Google AppEngine, Fusion Tables and ArcGIS Server. Much of what will be presented is baked directly into Arc2Cloud so if you’re interested to see another take on where Cloud based geoprocessing is heading, please vote up this talk