Starting in 2018, we set out to build a salary survey for the nonprofit sector. After thorough market research, the creation of a standardized title system, and drafting the survey questions themselves, we needed to present the survey in the most user-friendly way possible. Most importantly, we wanted to learn what our community expected to see after taking the survey.
We engaged Blue State, a leading design firm, on a full-scale redesign of our website, and in 2019, they designed the first version of our salary survey.
After an initial screening question to ensure we were collecting data from full-time nonprofit employees, the survey questions are split across three pages. The first page asks for information about the employee’s role, the second page asks about work location and details about their nonprofit employer, and all optional demographic information is requested on the third page.
At first, we didn’t allow for plain text entries in the job title field. We were focused on standardizing the data, so we required that one of our synonyms was selected from a dropdown. Paired with each job title is a description of job responsibilities, which helps to ensure that what someone is selecting matches our understanding of that specific title.
We ask for the survey participant’s most up-to-date annual salary, whether they supervise employees, how many years of experience they have in roles similar to the title they selected, and what their highest level of education is.
When the survey first launched, we asked about the employer’s location and focus area, as well as how many employees worked there.
On the demographics page, we asked about gender identity and expression, race and ethnicity, age, disability status, and whether respondents have served in the military.
After building an initial version of these form fields, we wanted to know if it all made sense. To gather this feedback, we put the salary survey in front of people recruited through an online platform. At the time, we used Validately, which was later acquired by UserZoom, which has now been acquired by UserTesting.com. (Ah, the wild world of tech startups!)
This user research platform allowed us to recruit users, record sessions, take notes, and export video clips that could be shared in internal research presentations. The core features of this platform have remained effective for us, and we continue to use it today.
From this first round of usability testing in February 2020, we were excited to learn that people found our survey intuitive and easy-to-use, and that they were comfortable sharing their information with us.
We also learned in these early days that people felt forced into selecting a title that didn’t match their expectations. This feedback led us to improve our job title field, allowing for entry of free text in addition to requiring selection of a synonym in our system. Through this process, we were able to retain a system of structured data while soliciting quick feedback about titles that were missing from our database, which enabled us to continue expanding the title system.
Given other developments in 2020, we evolved the “Employer Location” field to capture more nuance about remote work—asking for the respondent’s location in addition to the employer location, as well as which location their pay was based on.
Following this round of user testing, we made enhancements to our salary survey and began to design a more comprehensive "version 1" of a data visualization tool that could show someone how their salary fit within the landscape of all nonprofit employees who shared their job title. The first visualization included a comprehensive report in a long-scrolling experience.
After more design and development work, in June 2021, we ran another round of user interviews, this time gathering feedback about the visualization as well. We learned that people wanted to quickly see job recommendations based on the title they selected. We also got our first hints that there was an appetite for a “Career Path” tool that would show what someone could earn with more experience and/or higher-level titles. Further, people wanted to be able to change the number of years of experience they entered in order to see how it might impact compensation for the same title.
After another round of updates, we publicly launched the survey in July 2021. We rapidly began collecting great data and learning about our community.
The following year, Marian Blair, Idealist’s art director, took the design of this product to the next level! This round of polish made the visualization more interactive and easier to read. It also incorporated built-in methods for collecting ongoing feedback in the forms of a basic rating system and a feedback survey available in the UI.
With these updated visuals, we ran a round of user testing in October 2022. We received further affirmation of a desire for a career path tool, so Marian set to work designing what would become the most recent enhancement to the Nonprofit Salary Explorer.
Our career path module now lets people see right away what other titles might interest them as they progress in their career.
But there is, of course, an untold story happening behind all these updates. We have an incredible team of software engineers who have worked for several years to make this a world-class user experience. Let’s learn a bit more about the coding that went into creating this product.
Patrick Metzger
As the Director of Product, I help shape the vision and strategy across Idealist's digital products. Combining insights from data analytics and user research, I collaborate with our tech team to ensure we're building the best possible user experience for the global community of Idealists.