Building a heartbeat for the Pinecone Console most used feature with a real-time analytics and usage dashboard to help engineers monitor and troubleshoot their indexes.
Updated November 2023
Role: Product Designer
Collaborators: This was 1 of 3 projects I designed solo during my Product Design Internship. I was supported by and collaborated with 1 Principal Product Designer, 5 Full-Stack Software Engineers, 1 Product Manager and 1 Growth Product Marketing Manager.
Project Overview and Challenge
I was challenged to improve the adoption and user satisfaction of the analytics dashboard as my primary internship project. The dashboard provides metrics about a user’s Pinecone database instance (an index), including the number of items in it (vectors), reads, writes, latency, and errors. At the time, we had an analytics dashboard that was poorly received by users and internally. It was my job to learn why user’s need this dashboard redesign it. My new design introduced simplified graphs, animations and real-time graphs to give the Console a "heartbeat."
The project ended up expanding, and I had the opportunity to completely redesign the Index Details Page, which is where the analytics live. The team wanted a more flexible design for new features and product generations to be built on, starting with my analytics dashboard. This Index Details Page is the most used feature of the Pinecone Console, and my redesign has now served as the scaffolding for two additional new features: a database browser, and a namespace details panel.
I truly enjoyed the chance to dig into research and chat with people across the company, from engineers, to PMs, to customer support, to learn about what user’s want from an analytics dashboard and how I could bring it to life. I learned about interactive data vis best practices and explored different charting libraries. When I started this project, there was no PRD, just a blank slate, and it was incredibly rewarding to to collaborate with my team and PM to build the product requirements as we learned more about the problem.
User Research
At the outset, I had assumed that user’s wanted an analytics dashboard to look at for curiosity, usage monitoring, and digging deeply into their numbers for troubleshooting and optimization.
But from user interview and conversation with engineers, I uncovered that they wanted a way to quickly see if everything was working as expected.
As a fully managed cloud service, this makes a lot of sense. Our target user’s are technical, but they’re using Pinecone so they don’t have to spend much time or effort analyzing if their database is working. They wanted to know the product was operational, and check it’s pulse.
This idea of giving user’s a way to check the “heart beat” of their Index became the guiding principle behind project, and I frequently referenced a chart like this as our north-star.
dExcerpts from user interviews
I watched as a developer added data to their Pinecone database in a Python notebook, and then struggled to determine if it had worked. They expressed a desire to be able to visually see if the operation had worked without having to write additional code.
“Where is the button to reload the graphs, and why don’t they update automatically? I can’t tell if this is old data or not.”
“Is the dropdown selector with the units of time telling me how long of a period overall for the metrics, or the time-scale for the graph?”
“I just want to see the ‘Vector Count’ and if it’s going up or not.”
User Journey
I diagrammed a simplified version of the journey in FigJam of how new and existing user’s would use the Index Details page and Analytics to look for issues, troubleshoot, and check the heartbeat of their index.
Assessing the Current Metrics Panel
Armed with these new insights, I spent some time using the existing metrics panel myself. I wrote a few Python scripts interacting with the database to put some data on the graphs, and explored it’s functionality.
Original Graphs
- Refresh button seemed like a label for the time window dropdown since it had no outline.
- Lack of real time data made it hard to diagnose issues and see if the Index was working.
- No meaningful axes and unit labels.
- Unclear what “time window” selector controls. Is it time window, or unit scale of the graphs.
- Zero-states of the graphs reported a straight lines of the last reported data point, when in fact there was no data.
- Squished the units and lines, since the graphs were in a 2x3 grid, making it harder to see trends on larger time windows.
- Filtering required a dropdown for each graph, and user’s were confused when they couldn’t use the common design pattern of clicking on the legend to filter.
- Pale colors, leading to faint lines that were hard to see.
- Latency only showed 50th percentile, which user’s didn’t find meaningful.
Original Tooltip
- Blocked majority of graph you were looking at.
- Unnecessarily large, poor use of space, repeated information, and was not glanceable.
- Values were rounded to 4 decimal points, a precision we did not have.
- Digits and places did not line up, since the number were left aligned, and started at different places, causing the .
Original Index Details
- Status only marked with a color (not accessible) and not prominent
- Poor information hierarchy
- Index features and options were not glanceable, overwhelming, and visually fatiguing. Their labels were often redundant and had the same hierarchy as the information.
- Unclear if it’s active, is the database on and working properly?
- No warnings when pod is getting full
Before and After Comparison
Goals
With our north-star in hand, we developed the following goals:
- Heart beat for the Pinecone Console
- User’s can quickly know if everything is working by providing glanceable real-time data
- Redesign the graphs to meet best practices for data visualizations
- Flexible Index Details page to add new features
- Better latency metrics and organization
- Visualize Pod Fullness and Performance and provide warnings when it approaches certain ranges
Wireframing
I began by exploring different line charts, tooltips, and quick stats with low-fi wireframes. I wanted to explore straight vs interpolated lines, multiple lines intersecting, axes marks, and a bit of animation and hover interactions. I also wanted to explore the different time window selectors to make it more intuitive.
Line Charts Exploration (Medium/High Fidelity)
I began with a simple line with rounded points.
I added more lines and strong colors.
I explore if non-interpolated lines rather than curves would be easier to read, but it made it more difficult to track each line.
I experimented with different line weights before settling on a fairly light one. Heavier ones were easier to track across the graph, but lost precision and felt cluttered.
I added points to mark the actual data points vs the interpolated curve.
A few different ways of displaying the legend.
Exploring hover interactions on a specific line, fading other lines away, and highlighting it’s values on the axes.
Legend hover interactions for highlighting a line.
Filtering and soloing a line by clicking on a legend item.
Exploring a line chart where the point indicator only appears on hover, which is less cluttered, but also less precise.
Line Charts vs Bar Charts vs Stepped Charts Exploration
Originally, we had decided all the graphs should be line charts. But after chatting with engineers, I learned that this indicated that all of data was continuous, but in reality most of the data was discontinuous, so the line graphs were misleading. On top of that, the line graphs made it difficult to distinguish overlapping data which was quite common.
I did some benchmarking and research and determined that the data vis best practice is stacked bar charts.
A new chart with a new legend that fits, and a (temporary) new color scheme.
I explored every possible bar spacing and bar width to find the one that was easiest to analyze and spot trends or anomalies.
I wanted to see if adding a vertical gap between stacks would help. It didn’t.
I explored dozens of color schemes.
And then gave the chart the full vertical width, to give everything some more room to breathe and show more data at once. The spacing of the bars was still off.
Final Charts
Finally, I settled on a color scheme for the bars that had just enough contrast for each item. And I found the right combination of width to gap to make it comfortable to glance at and analyze. I found that a gap of about 15% the width of the bar was comfortable.
For the errors section, I adapted the same color palette but rotated the hues.
Time Window Selector Exploration
I wanted to avoid a dropdown so user’s could quickly skim different time windows, and make it abundantly clear with the copy and design that this was a window selector. I made the refresh button more of a button, but about half way through designing it, we had confirmed we’d be able to stream the data in real time so we could remove it.
Final Time Window Selector
I flipped the deactivated vs activated colors at the last second, so lighter is now active and dark is disabled which was more intuitive. I settled on the word “Past” instead of “time window” since a user looking at is likely to interpret it as “data from the past 12 hours” which is easier to understand than “data within a time window of 12 hours”. It’s more obvious and more clear, and shorter!
Request Latency Exploration
My PM and I had two big changes for the Request Latency graph.
Our big breakthrough was that instead of 1 chart showing 1 latency metric, where each line was 1 of the 6 “Request Types”, we’d make it 6 charts that could be toggled through, with each chart being for 1 of the 6 Request Types. Simplifying the chart would mean we could show more latency metrics on each chart, making it far more meaningful. We arrived at this because in looking at the way people used request latency, they generally only cared about one Request Type at a time. We also switched from 50th Percentile Latency (P50) which turned out to be a bad metric for measuring latency, to showing 90th, 95th, and 99th Percentile Latency.
A bar chart didn’t make sense for this graph, since the three values don’t sum to a meaningful number. I experimented with an area line chart, since the P90 will always be less than the P95 which would be less than the P99, and the relationship and difference between the latency metrics for any given point is meaningful and provides context for the severity of the latency.
Final Request Latency
The area graph had the same issue with the line charts, which is that it represented discontinuous data as continuous, so we settled on a stepped area chart to display the data.
Performance
Finally, we combined the line chart and the vector count chart to create a performance chart to help user’s see how their vector count is influencing their fullness (which is not always 1 to 1), and added bands to warn users when their performance may degrade if their index get’s too big for their plan.
Tooltip Exploration
To start, I wanted to get each piece of data stacked on top of each other. Then I added some structure and hierarchy and experimented with the labels. I also wanted to reduce the the height, so I moved the labels around and combined them into a suffix.
During my research, I read that it’s easier to scan numbers if they are right aligned because their digits line up vertically. I explored different ways of right aligning the numbers.
I settled on a layout with the label on the left and the value on the right, with lower hierarchy units since they are all the same. I added some color and tried a 2 column layout, but decided it would cover too much of the chart. “Describe Index Stats” was causing some trouble since it was long and pushed the content over.
Final Tooltip
We ended up removing “Describe Index Stats” for unrelated reasons, which helped tighten the width. And I condensed the card so it would cover less of the chart, and rounded only the top and bottom color tags to match the bars. I also inverted the order of the items so the tooltip would match the order of the bars.
Index Details Exploration
Our north star was to give the Console a heartbeat, so user’s could quickly see if everything was working. So halfway through the project, I was empowered to redesign the Index Details card above the Analytics to achieve this.
From my user interviews, it was clear that “Vector Count” had a special importance for users to assess if their Index was working. If they added items to their Index, and the number went up, it meant it worked. And this was the most common thing they’d check in the Index Details Page.
I decided to move the Vector Count graph from the analytics dashboard in the Index Details summary, so that it was quickly visible. It would update in real time, and have a pulsing endpoint if the index was active with no errors. This would be the heartbeat.
I started by restructuring the card. I converted the labels to descriptions and gave everything a bit more structure and room to breathe. I also added icons to help make it glanceable, and added “Active” to the indicator to make it more accessible and prominent.
I experimented with chips, but ultimately they felt too cluttered and seemed like they might be buttons since we use chips as buttons elsewhere.
Since some of the info in this panel would be changing when V4 of Pinecone launches next year, I explored a few modular designs with panels that could be swapped in and out over time or depending on which version of the product a user is subscribed to.
These were looking better, but the Deployment URL would often truncate, which we wanted to avoid.
This was close, but it felt a bit cramped, and the sections were not very flexible or modular.
Final Index Details
Our final version has three sections: details, status, and performance. Each one can be swapped and adjusted over time, and the vector count and Status indicator are prominent and provide signs of life to a user quickly scanning it.
Final Design: Putting it all together
Before and After Comparison
2 Wins
Watching a user check their index’s heartbeat
About a month after wrapping up my internship, I was working on a coding project with a friend and he wanted to try out Pinecone to power it. As he was coding, I heard him say “hmmm…I’m not sure if the items went into the index”, and I silently let him figure it out so I could see if my hard work had paid off, and then about 3 minutes later he said excitedly “ohhhhhh…look, IT’S WORKING” and flipped his laptop screen around to show me that the Vector Count on the Index Details page was going up “LIVE”. I couldn’t have scripted it better if I tried, and watching it happen reminded me why I love making products: to help people make things and delight them along the way.
Laser focused on what matters
I had a very clear north star, to make it as a quick and easy as possible to see if your index was working, and I think I achieved it. It’s far from perfect, nothing is, but it’s a marked improvement that spotlights the most important information, makes it clear if your index is active, and I was even able to work with the engineers to provide real time data and live updates by default which improved the user experience, and the constant updates makes the console feel alive and exciting.
2 Improvements
Interaction design
In the low-fi wireframing stage, I explored a lot of different interactions to help user’s explore and understand their data. I’m proud that the most essential ones like tooltips, and filtering by clicking on the legend shipped, but the opportunities in data visualization for complex and deep animations and interactivity are enormous, from dragging on the graph to select a time window, to drawing trends and making predictions. Ultimately, these were outside the scope and wouldn’t have moved the needle on our design objectives significantly, but if I had more time or get the opportunity to work on a similar project, I would love to explore it.
Making it responsive and handling zero states
Although the Pinecone Console is not meant to be used on mobile, that may not always be the case and I think the charts will need some revision when that happens. The interactions will need to be modified to work with dragging rather than hovering, and we won’t have the width that we do on desktop (although on smaller desktop screens we already have this limitation), which will create some new design problems to solve like handling the legend and adjusting the time window. Maybe we could explore adding an ability to adjust the scale of the graph to compress the data for a smaller screen. Regarding zero states, we settled on showing blank graphs when there is no data. Ideally, I think we’d want a message to display indicating there’s no data and providing hints on how to resolve that, which is something I prototyped. Ultimately, we had a technical limitation of the way the backends sends the data that forced the blank graphs, but I still think it’s worth calling attention to.
2 Takeaways
The value of finding a metaphor
I identified the heartbeat metaphor early on in the problem discovery process, and having that metaphor throughout the entire design process proved incredibly helpful. When I was communicating and pitching my ideas and solutions to my team, it helped everyone concretely point to we working on. And through the process, it served as helpful sanity check for making decisions: “does X help create a heartbeat, or not?”. I thoroughly believe that product design is as much about communicating problems and solutions as it is about identifying and solving them, so I enjoyed developing that skill and learning a new technique for communicating about design.
It’s a team sport, and communicating early and often is everything
When I quickly showed my nearly final designs to the engineer that would be developing it a couple days before I finished, he jokingly said “This is awesome, Reese…now I have to figure out how much of it is possible”. And then I reminded him that before I started designing it, he showed me the charting library he wanted to use for it, Nivo, and I told him that throughout the design process I had been referencing the Nivo documentation and examples to make sure that everything in the design could be done out of the box.
He was incredibly thankful and told me it would cut down on the development time significantly. I was glad to help and it reinforced the importance of communicating with stakeholders, from engineering, to product, to growth and legal, throughout the design process to ensure it goes smoothly. Making a product is a team sport, and that anything I can do make my team’s life easier will ultimately result in a better product.