As the 2020 U.S. election cycle hit overdrive, demand on Civis Analytics’ systems reached historic levels, resulting from a sharp increase in jobs — executable tasks in Civis Platform such…
Happy New Year from the Civis Platform team!
Today, we’re excited to announce native support for PostgreSQL on Civis Platform. Harnessing the power of Postgres, Platform users can now run high-performing, relational apps within their environment, as well as utilize geospatial data to create a more complete and rich view of customers, constituents, and communities.
The Power of Postgres and Civis Platform
Since its inception, Platform has run primarily on Redshift. This has served most use cases for our clients; however, we wanted to support additional features and improve the overall Platform experience in ways that our current client database infrastructure wouldn’t allow. We realized that there are multiple database solutions that data scientists and analysts use in their work, and adding Postgres as another database option was the next logical step forward in our goal to provide a more flexible solution that provides data scientists with a valuable toolkit.
We are particularly excited about two key enhancements to Platform that Postgres enables:
Geospatial data support: Perhaps the most front-and-center feature now available through Postgres compatibility is our support for geospatial data. You now can store and query geographic data types, using the popular PostGIS extension as the backbone.
We’ve added import and export jobs for shapefiles and geodatabases as well as first-class API endpoints for geocoding, which give you many options to work with coordinate-based data. Finally, as Platform supports using open source packages in your code, you can generate maps using the package of your choice. We’ll share an interesting use case about these new features later in this announcement.
Database and App Performance: Postgres and Redshift complement one another nicely: Postgres is row based and works well for backing apps that process transactional data, while Redshift is column-based and is optimized for queries on large datasets. Postgres’ flexibility compared to other relational databases make it the gold standard for anyone looking for a row-based, relational data store. With Postgres functionality, Platform users can now decrease the latency of their Services by backing them up with an optimized, relational database.
Geospatial Support in Action: Census 2020
Much of what we’ve learned around our customers’ use of geospatial data stems from work we’ve done with local governments, including the City of Houston. Local governments are always interested in how different issues or policies impact different neighborhoods or communities, and mapping data spatially is important to understanding the unique needs of each neighborhood.
For example, many cities are currently planning their outreach campaigns to make sure that all of their residents respond to the upcoming 2020 Census. Using PostGIS, we’ve started to help cities combine US Census Bureau data with their own data sources to develop geographic maps of areas likely to have low census response rates, often called “hard to count” areas. This type of data allows municipalities to identify residents who are unlikely to respond to the Census and how they are distributed across a given city. Municipalities, foundations, and other groups can then use this information to understand where outreach could be more effective, and what message and type of outreach will be most effective at increasing response rates. (For more details on our approach to the 2020 Census, check out our whitepaper on the topic.)
Our mapping of the San Francisco Bay Area census tracts. Using PostGIS, we can see which tracts have a higher likelihood of not responding (dark purple areas).