Enabling enterprise sales reps to surface answers and content through natural language, without filing a ticket or waiting on the enablement team.
Author Carlos d'Abreu
Status Shipped
Product Mediafly AI Copilot
Phase MVP / v1.0
Team PM, Eng, Design, CS, Sales
Problem
Enterprise sales reps were losing time mid-cycle looking for answers to product, pricing, and configuration questions. The existing path was to file a support ticket or ping an SE, both of which introduced delays that disrupted live customer conversations. Support teams were fielding a high volume of repetitive, self-serviceable questions that didn't require human judgment.
Sales reps needed answers in under 60 seconds during live customer conversations. The average ticket resolution time was measured in hours, not minutes.
Goals
- Reduce inbound support ticket volume from sales-originated queries by at least 40%
- Enable reps to surface accurate answers from Mediafly's content corpus without leaving the app
- Build a feedback loop that improves retrieval quality over time
- Ship an MVP fast enough to influence two active renewal conversations
Non-goals (v1)
- Generative content creation (write a proposal, draft an email) — deferred to v2
- Voice input — deferred pending UX validation
- Proactive nudges and recommendations — deferred to v2
- Cross-tenant knowledge sharing — out of scope for security reasons
Users
Segment | Primary need | Success looks like |
Account Executives | Answers to product and pricing questions during live calls | Answer surfaced in under 60 seconds, no ticket filed |
Solutions Engineers | Technical configuration and integration details | Accurate spec retrieved without escalating to product |
Customer Success | Self-service for common renewal and expansion questions | CS reps handle tier-1 questions without PM involvement |
Solution overview
A retrieval-augmented generation (RAG) pipeline embedded in the Mediafly app surface. Users ask a natural language question; the system retrieves relevant content chunks from a curated corpus (product docs, pricing sheets, battlecards, internal FAQs), ranks by relevance, and synthesizes a grounded answer with source attribution.
Architecture decisions
Decision | Choice | Rationale |
Retrieval approach | RAG over fine-tuning | Faster to update corpus; lower hallucination risk on structured data |
Action boundary | Suggest-and-confirm for all outputs | Protects against wrong outputs in live customer contexts |
Feedback loop | Thumbs up/down on every answer | Surfaces low-confidence retrievals for human review |
Observability | Full query and retrieval logging from day one | Silent failures in agentic systems are the worst kind |
Corpus scope (v1) | Curated internal content only | Controls quality; prevents hallucination from unvetted sources |
Requirements
Requirement | Priority | Notes |
Natural language query input in-app | Must have | No new window or context switch |
Answer synthesized from retrieved chunks with source citation | Must have | User must be able to verify the source |
Relevance threshold — no answer surfaced below confidence floor | Must have | Show "I don't know" rather than a low-confidence answer |
Feedback mechanism on every answer | Must have | Thumbs up/down feeds review queue |
Admin corpus management UI | Should have | Enablement team can add/remove/update documents |
Query analytics dashboard | Should have | Tracks top queries, low-confidence rate, feedback scores |
Suggested follow-up questions | Nice to have | Surfaces related questions the rep may not have thought to ask |
Success metrics
Support ticket reduction | Sales lift (copilot accounts) | ACV influenced |
55% | 73% | $2M+ |
Achieved in production | Increase in closed sales | Via value engineering tools |
Target answer confidence floor: 80%. Target query-to-answer latency: under 3 seconds. Feedback loop review cadence: weekly by enablement team.
Risks and mitigations
Risk | Mitigation |
Hallucination in structured data contexts (pricing, specs) | Confidence floor hard-coded; answer shown only if retrieval score exceeds threshold. Source always shown alongside answer. |
Stale corpus undermining answer quality | Admin UI enables enablement team to push updates without engineering. Reviewed weekly. |
Low adoption if first-run answer quality is poor | Piloted with 3 power users before GA. Feedback loop seeded with 50+ curated Q&A pairs before launch. |
Security: sensitive deal data in query logs | Logs stored in isolated tenant partition. No cross-tenant query visibility. Reviewed with legal pre-launch. |
Launch plan
- Pilot with 3 internal power users (2 AEs, 1 CS) — 2 weeks
- Seed corpus with 50 curated Q&A pairs and top 20 support ticket categories
- Internal beta to full sales team — 2 weeks, feedback loop active
- GA with CS-led enablement session and in-app onboarding tooltip
- 30-day post-launch review: ticket volume, feedback scores, query analytics
Made with Bullet