Designing for Transparency and Justice
Oath
Timeline: July 2024
Role: Product Designer
Core Responsibilities: Generative User Research, UX Design, Product Strategy
Key results: Alpha release in 2024, pending metrics
Team: Founder Jonathan, Product Manager, and Lead Engineer
Tool: Figma
Oath is a consumer-facing identity infrastructure for fraud prevention and a contract management platform. Oath’s founder hired me as their first product designer to help design a key experience for Oath’s alpha release: the dispute moderation flow.
Entrusted with great responsibility, I quickly became the de-facto Subject Matter Expert on the moderator experience and led the entire design process from scoping the project and conducting user research to delivering development-ready designs.
Armed with a little bit of naive optimism, I set out to solve this problem:
How might we restore mutual trust, collective honesty, and civil justice by turning a customer service nightmare into an opportunity to strengthen collector communities?
The problem: Our product’s origin in toy trading groups
Bad experiences shopping on social media lead to loss, distress, and hyper-vigilance
If you’ve tried buying stuff on Facebook Marketplace, you probably don’t just trust anyone on the platform. You and everyone know agreements on social media are more like “pinky promises” which don’t have any customer protection.
When disputes do unfortunately arise, both buyers and sellers face obstacles like:
Lack of formal agreements between parties
Minimal protection from social media platforms
Biased treatments from payment platforms like banks or PayPal
Social media fraud has very real economic consequences
In just the first three quarters of 2024, over $1.7 billion in social media fraud losses were reported from 120 thousand cases in the U.S. – reflecting a staggering 81% (!) compound annual growth rate over recent years.
What if…?
we reimagined a new online experience with transparency and justice at its core
Product vision of Oath
Oath as a community of traders operated on trust and open communication
Before traders (buyers and sellers) begin their social media buy/sell/trade experience, Oath acts as a figurative security officer that matches a user’s identity with their account. When they join the Oath community, they’re equipped with Oath’s easy-to-use, templated contracts to formally document the buy/sell/trade details previously agreed upon.
If only everything goes well perfectly all the time.. Misunderstandings can occur even among well-meaning buy/sell/traders. This is where the dispute moderation process (what this case study is about) comes into play. An Oath moderator basically serves as a private judge, determines which party is favored, and facilitates the process of negotiating compensation.
More context
We walk the walk – protect users with real protocols
While the dispute moderation flow isn’t part of the ideal “golden path” for Oath users, it’s still a critical safety net for our early adopters/biggest advocates and a key component to building strong a brand reputation – of being reliable – and user loyalty.
Qualitative research insights on user needs
Buy-Sell-Traders need an impartial facilitator
Through secondary research and analysis of countless anecdotal stories, I identified two key expectations that traders (buyers and sellers) have for the platform:
Quick responses with transparency throughout the resolution process.
Fair and unbiased treatment—avoiding any reputation for favoring buyers or sellers.
These insights shaped the core principles of Oath’s moderation flow, ensuring it was designed to instill trust and confidence in all users.
Moderators are passionate allies that Oath empowers
From conducting unmoderated user interviews with 5 moderators, I discovered the following:
A strong sense of justice: Moderators want to ensure fairness and order in their communities.
Limited enforcement tools: They often feel powerless to hold scammers accountable beyond issuing bans.
This understanding guided my approach to designing tools that would empower moderators, giving them the resources needed to facilitate fair and effective dispute resolutions.
3 key design solutions
As a starting point for myself, I prioritized designing the dispute investigation process from the moderator's perspective first – validating my approach with real user feedback before shaping the multi-sided dispute reporting process with all three user groups. The rest of the case study goes over my design rationales for:
Component-level interactions for the investigation form.
Feature-level communication mechanisms for dispute resolution.
Product-level information architecture to seamlessly integrate these features into Oath.
Form follows function
Some flexibility required in dispute investigation
Need #1
From talking to moderators, I learned that initial dispute reporting from one side often doesn’t always cover all the bases for any judgments to be made. Moderators frequently need additional information gathering such as:
Requesting proof, like screenshots of conversations.
Following up with complainants to clarify their expectations.
Reaching out to the accused (and sometimes suspected scammers) to hear their side of the story.
Conducting video calls for added clarity in complex cases.
In other words, Oath’s dispute investigation process needs to strike a careful balance! The platform should provide enough structure to streamline the workflow while remaining flexible, allowing moderators to adapt their approach and thus the dispute case outcome based on the specifics of each case.
Progressive disclosure and opt-in design to streamline dynamic, multi-stage judging process
Solution
Forms are not the most visually exciting things to design, but functionally, they offload the complex decision-making process of dispute moderation from our moderators’ minds to conserve their energy while retaining full control to adapt to specific cases.
These screens and the general logic below have been validated by our Mod interviewees, but the alpha release will truly test the flexibility of this form in handling real disputes and providing real user feedback on the front lines!
Outcome #1
Minimum requirement met = false (Default state)
The default state “No” for the minimum requirement of proof question is intentionally designed to ensure that sufficient evidence for a valid complaint is provided before any judging takes place.
And once minimum proof has been submitted (and ideally from both parties), the moderator has the option to flag any inconsistent or potentially altered evidence (see outcome #2 vs #3 below), as there have been instances of photoshopped screenshots reported by our interviewees.
Outcome #2
Minimum requirement met = true (User opt-in)
Inconsistent evidence = true (User opt-in)
The “Flag inconsistent evidence” checkbox and “Decide in favor of” judging panel both utilize progressive disclosure based on user input to enable three distinct initial investigation outcomes.
The general decision-making logic has been validated by our Mod interviewees, but the alpha release will truly test the flexibility of this form handling real disputes and provide real user feedback from the ground.
Outcome #3
Minimum requirement met = true (User opt-in)
Inconsistent evidence = false (Default)
For completeness sake, the corresponding buyer/seller view for reporting dispute is designed with the goals of:
quickly identifying and contextualizing the conflict,
encouraging succinct reporting in the text box
Facilitating easy media uploads
After translating the details of the judging process into design, it was time to zoom out and design for the interactions between the buyer, seller and moderator during the dispute process. What would a helpful back and forth between users look like?
Opening up conversation
Need #2
Multi-way communication requested for dispute resolution
Multiple moderators I’ve interviewed repeatedly expressed the value of having both seller and buyer dispute to help them construct a cohesive narrative.
Thankfully, our tech stack was already built with communication needs in mind, so there was no surprises for our engineering team.
Ideation
Transparency-first approach to post and reply interactions
Aligning communication component with Oath’s source of truth mental model
In comparing the different communication-related UI components, I prioritized the user value of offering transparency to:
help seller and buyer by allowing them to see any updates on the dispute, thus fostering trust
give moderator full context into the interaction between traders to help them remain impartial,
offer the wider collector community a lesson on what kind of practice doesn’t work.
Scrapping and exposing old submission form as composer
While Oath’s dispute process must offer more structure than the current experience in free-form messaging apps, at first it wasn’t clear if the reporting form should be a stand-alone interface like most customer support request forms today.
I decided on the design of an embedded/integrated composer as opposed to a stand-alone submission form – because of the advantages of:
signifying open-communication,
enabling context referencing and
minimizing cognitive load
Reply interaction iteration: nested replies vs individual messages
Working around an existing activity feed with timestamps, the design challenge for me was to integrate and present users’ communication dialog in relation to other Oath status updates.
Iteration #1: nested replies (discarded)
After sketching out a nested replies mechanism (left video), I decided to use the mechanism of composing individual reply messages in a separate composer (right video) which aligns better with the rest of the individual activity updates on Oath and enables clearer sorting of the timeline.
Iteration #2: individual messages (kept!)
Solution
Contextual, multi-state message composer for reporting, posting, and replying
Combining the exposed reporting form with the reply message composer, we have a dynamic multi-state message composer adaptable for dispute reporting, update posting, replying, leaving reviews, and writing thank you notes.
When we release the product, I’ll monitor the ratio of people using the report button toggle in composer vs the report button in the header to iterate on the appropriate entry points..
Situating composer
Need #3
Making sense of adaptive composer hierarchically
As I was getting the components of the composer and investigation form figured out for our alpha release, the long-term roadmap for Oath’s functionalities was still transforming in our founder’s mind.
I invited our founder to workshop together and clarify the user flow he envisioned for traders navigating the Oath contract page, which enabled me to design a strong (and hopefully future-proof) information architecture and place the composer strategically.
Ideation
Information architecture sketches balancing understandability and discoverability
Assessment of user flow and component value
After defining the distinct roles of the Offers tab and the Activity tab, it became clear that the buy/sell/trade-related correspondence belongs in the Activity tab!
UI layout sketches visualizing different Information Architecture models
After comparing the 3 sketches, the option where the composer is right underneath the secondary navigation tabs offers optimal discoverability for ease of use and understandability of its relationship with other activities.
Solution
Oath page > Activity > Composer
I reorganized Oath’s main interface with a clearer hierarchy, situating the composer under the Activity tab, offering clear affordance of communication capability within an active Oath case and allowing direct initiation of the dispute investigation process.
Putting things together~
finally!
Trader view – Oath page
Trader view – Reporting Dispute
Trader view – Reporting Dispute Expanded (Contextual Composer)
Moderator view – Investigating Report
Moderator view – Investigating Dispute / Judging
Moderator view – Investigating Dispute /Judging Expanded (Contextual Composer)
Community-approved concept currently in production
Impact and Next Steps
I was very fortunate to get early feedback from the amazing Lego moderator community (spanning the US, UK, Canada, and Jordan) to:
validate the design solutions for the dispute moderation experience,
address blind spots before the alpha release.
While our team anticipates relatively few disputes (fingers crossed!), we aim to achieve high dispute-related task completion rates, reduced resolution times, and high user satisfaction. To gather more qualitative insights, we have established Discord and Facebook channels for direct user feedback, which will inform post-launch improvements.
In the future, with additional funding, I would like to conduct in-depth usability testing to further refine the moderator experience! As Oath scales, I would also explore enabling collaboration features for moderators and revisiting the private note toggle I set aside earlier.
Dare to be a newbie and embrace my designer voice
Retrospective
Initially, I felt a sense of imposter syndrome worrying that my lack of personal experience as a toy collector and moderator might hinder my work. Luckily, I quickly overcame this by learning directly from users—similar to how I approach designing for specialized B2B products.
Working on Oath has once again shown me the power of design in shaping a product in its infancy. And I owe many thanks to Oath’s founder for believing in my ability and fostering a culture of curiosity and compassion. It's truly an honor to have contributed to Oath’s early journey.