In the wee hours of an October morning, I rested my forehead on my desk. The data in the database wasn’t matching the data in the CRM. There were duplicates in the list of targets. My laptop was so slow. And the report was Still. Not. Finished.
This stuff happens to untold numbers of data scientists every day, and solving it is what Civis Platform is all about. I’ll pause here for a second. If you’re reading this and thinking, “yeah, yeah, just let me try it out,” then feel free to stop reading and sign up. That’s the whole point. We’re now offering a free trial of Civis Platform.
See, data science is hard, but there’s “good hard” and there’s “bad hard”.
On that October morning, hundreds of people were relying on my data work — that was a good feeling. But I didn’t have the computing power I needed, the database skills or the sysadmin skills to get my work to all these people on time. That was a bad feeling.
Civis Platform is for data scientists who want to scale up their work and put it into the hands of decision makers. It’s the best way to go from building models and visualizations to building data science workflows, tools, and applications. That’s because Civis Platform lets data scientists work however they want — in Python, R, and SQL — but backed by scalable cloud resources, third-party data libraries, and a robust API, all while handling all the security and credentialing. It means more “good hard” and less “bad hard”.
Though my story comes from doing data science on a political campaign, it happens to data scientists doing everything under the sun. It might be that scoring a boosted tree model on millions of records takes way too long, or that collaborating on workflows with coworkers means resorting to emailing files, or that serving up a simple web app means waiting for attention from the IT team.
We know these struggles because Civis Analytics is a company of data scientists from every possible background — physics, finance, consulting, politics, marketing, linguistics — and we are using the platform every day. Our goal is to make all Civis Platform users say, “I wish I had this at my last job.”
That’s why we say that Civis Platform is by data scientists, for data scientists.
In the last couple of years that Civis Platform has been around, users have run more than 4.5 million jobs. That’s millions of database imports, machine learning models, SQL scripts, Python scripts, R scripts, Salesforce exports, data appends, matching jobs, geocoding jobs, Google Sheets exports, and many more. To me, the coolest part is that the vast majority of those runs were in automated workflows and ran while the data scientists who set them up were working on their next big project.
What are some of the amazing workflows we’ve seen?
- Build an app for exploring model scores that kicks off Python scripts to generate new models as the user explores an HTML frontend.
- Simultaneously build and score hundreds of gridsearched models from scikit-learn for dozens of different dependent variables to generate the best possible list of marketing targets.
- Pull together the daily master report for a company’s marketing work, drawing data from multiple CRMs and databases and transforming it with SQL.
- Match and append from the Civis database of all American consumers to better understand and communicate with customers.
You can do all of these things, and an infinite number of things we haven’t thought of, in the Civis Platform free trial. We know that for data scientists, showing is better than telling.