home

Welcome to the soc-maps wiki!

Toward a Coherent System of Managing and Publishing Geospatial Information

Managing Spatial Data Repositories -- Initial Approach and Legacy Issues

CCCS maintains separate geospatial data repositories for our company (comprised entirely of publicly-available data) and for our clients (which typically include a mixture of of both public and proprietary data sources).

CCCS' public data repository is currently managed via GitHub as /cccs/soc-maps/. This repository was established with the initial objective of being a place for developing a web application to showcase our capacities in spatial analysis and the production of cartographic data visualization in support of social impact assessment. The goal was for this web application to be presented through our Django web framework, hosted at crossculturalconsult.com.

CCCS' early development work utilized GitHub to version-control both the web mapping software application as well as the spatial data that we intended to referenced by that application. While this approach made it convenient for sharing both the data and the code base with our various team members, both CCCS' IT expert, Paul Whipp, and our GIS consultants, Kartoza Pty., recommended against this 'combined' approach. The Git version-control system is non-optimal for management of spatial data primarily because repository size for such data tends to grow too large for Git to be effective [e.g., one tends to start encountering timeout errors when cloning and pulling larger repositories, and the system may not be able to manage any single file over 2 gigabites in size (such as a a dump of a spatial database)].

Following the advice of our consultants, CCCS adopted an alternative system for sharing spatial data using btsync. While this approach has the advantage of allowing us to share larger-sized repositories of spatial data outside of Git, it has the disadvantage of creating parallel data sources (i.e., the shared btsync repository on the client side and the sever-side 'production' repository. Another disadvantage of the btsync approach is that data are not version controlled. We therefore view the btsync data-management approach as a 'temporary' solution.