Great Expectations (GX) is a data quality tool that helps organizations create flexible rules and apply their industry expertise to test on their data. GX had a large following of users that enjoyed the flexibility and versatility of a free open source software that allowed users to define and implement rules or “Expectations” that runs and tests their ever growing datasets to ensure that their data are up-to-date and correct to make effective business decisions.
Product Designer, UX Researcher & Designer, Visual Designer
The product was launched in an open beta state, allowing users to sign up and start using GX Cloud immediately. However, we quickly realized that users were not completing the desired user journey and were dropping off before experiencing the product's full value.
CONTRAINTS
The product relied on open source offerings that had required a steep learning curve to understand and fully take advantage of GX's flexible “Expectations” to run against their data sets. It was also quite simplistic to a fault, most users came from our Open source offering that had much more complex workflow and capabilities that were not offered in the initial GX Cloud release.
SOLUTION
Allow users to test out GX Cloud application by offering a demo of the product so users can understand the suggested workflow, features, and value that GX can offer.
GOAL
Increase users adoption rate and use of our Cloud product by allowing users to run and apply Expectations to sample dataset to better understand how GX Cloud can help their organizations.
GX had announced our early beta Cloud offering early during Q1 of 2023. A few months after release our team needed to understand how users were interacting with our Cloud product. We were made aware from our devops team that we had Sentry replays to better understand our user behaviors.
After a few weeks of analysis, we observed almost all the users that had signed up for our product and we quickly discovered a core issue that affected our new users and significantly increased users drop rates.
Having a more clear understanding of our low adoption rate we began to discuss what potential ways GX can quickly showcase the value or "Aha moment" to users. Users would need to achieve this point in the journey to be able to fully understand the value our Cloud product offered.
We quickly spent some time researching both our competitors along with other relevant and similar products within the industry. This was important as the need for data quality has only grown in importance over the past decade. After analysis of similar products we noticed some similarities and minute differences in approaches from product to product.
After some discussion and understanding of the various approaches we reviewed this with our Product team to better improve our MVP flow to reduce drop off rates to allow users to better see the value of GX Cloud. Since I was involved in the research phase I had a vague idea of what I thought was the best approach and reviewed with our product team to confirm that the team was on the same page. Our team needed to understand why users were dropping off and needed to take constraints into consideration:
The main goal is to allow users to successfully navigate onboarding and being able to test out the "Expectations" that GX Cloud offered. Code name Demo Data, our team quickly iterated on changes to the MVP flow that would allow users to get to GX Cloud's value moment. In my 3 years of working at GX this was by far one of the most successful feature releases.
The release of Demo Data was a huge success overall. We successfully increased users' completion rate by 300%, an increase from around 30-40%. This was a huge feature release for us as we were able to observe more users complete the intended journey while being able to better understand critical pain points.
Alongside the MVP improvements, we were working hard on a way to allow users to leverage AI and have Expectations recommendations based on their data. Leveraging user interviews it turns out that the answer was surprisingly simple: "We just want to apply basic tests on our data." Users were able to tell GX Cloud that they would like to apply recommended Expectations as an option to be able to quickly see simple tests applied. Tests include simple topics such as missing data, volume, and detecting outliers.
Thank you so much for reading about this case study! I very much appreciate the opportunity to work on such an amazing product, taking part in so many different types of work and learning so much along the way. Unfortunately our company was reduced in half due to the economical environment.
On April 30, 2025 I welcomed my first born son Ari to the world! I was able to take some extended parental leave and learned so much on the difficulties and joys of being a parent.
