How Social Department built culture-aware video discovery
Social Department uses AWS Bedrock, OpenSearch Serverless, and custom pipelines to deliver cultureâaware video discovery through QuickSearch actions that surface shareable, quotable, memeable, and trending clips.

How Social Department Built CultureâAware Video Discovery
Entertainâment marketers juggle massive video libraries and tight campaign deadlines. Traditional semantic search falls short because it can only answer concrete, describable queriesâ"find every coffee mug"âwhile marketers need judgmentâbased answers like "which moment will go viral?" Social Department tackled this gap by turning editorial intuition into executable retrieval pipelines, delivering a set of QuickSearch actions that surface the right clip without the user having to articulate it.
---
1. Why Plain Semantic Search Wasnât Enough
- Judgmentâdriven queries â Marketers know what will travel, not what to type. The skill lies in audience perception, not just content description.
- External trend signals â Audience conversations evolve hourly, whereas a video library changes only with new releases. A discovery system must continuously ingest external buzz and align it with the static library.
These constraints forced Social Department to separate content understanding (whatâs in the video) from trend acquisition (whatâs happening outside the video) and only merge them at request time.
---
2. QuickSearch: Four ActionâOriented Retrieval Paths
| Action | Goal | Retrieval Strategy | |--------|------|--------------------| | Shareable Scenes | Find selfâcontained, highâimpact scenes (climaxes, confrontations) | Sceneâlevel reasoning models rank longer segments, biasing toward narrative arcs. | | Quotable Quotes | Surface punchy dialogue suitable for soundbites | Transcript indexing + popularity signals (social mentions, subtitle spikes) filter short clips. | | Memeable Moments | Identify single frames that can become memes and autoâgenerate captions | Frameâlevel visual analysis selects standout images; a caption model grounds text in the frame context. | | Trend Align | Match current external trends to relevant library moments | Realâtime trend feed (Twitter, TikTok, news) is scored against metadata; a crossâmodal ranker surfaces matching clips. |
Each action runs its own retrieval pipeline, applying editorial criteria defined by Social Departmentâs content experts.
---
3. AWSâPowered Architecture
Content Understanding Pipeline
- Ingestion â Video files trigger an AWS Step Functions state machine.
- Processing â ~24 Lambda functions extract audio, subtitles, visual embeddings, and scene boundaries.
- Modeling â 12 Labsâ MarengoâŻ3.0 and PegasusâŻ1.2 run on Amazon Bedrock, providing sceneâlevel reasoning and caption generation.
- Indexing â Enriched metadata is stored in Amazon OpenSearch Serverless, enabling fast vector and keyword queries.
Trend Acquisition Lanes
- External APIs (social platforms, news aggregators) feed a streaming ingest job.
- Trend items are normalized, vectorized, and stored alongside timestamps in a separate OpenSearch index.
QuickSearch Job Layer
- When a marketer selects a QuickSearch action, a Lambda orchestrates:
1. Pull the relevant content embeddings from the library index. 2. Pull current trend vectors (if applicable). 3. Apply the actionâspecific ranking model (hosted on Bedrock) that respects editorial guardrails.
- Results are returned to the UI in under a second, ready for human review.
Responsible AI Controls
- Data residency â All customer video and metadata stay within the AWS boundary; Bedrock Guardrails screen prompts for promptâinjection attacks.
- Humanâinâtheâloop â Generated quotes are crossâchecked against episode transcripts; a marketer must approve any clip before publishing.
- Audit logs â Every retrieval request and model inference is logged in CloudWatch for traceability.
---
4. Practical Takeaways for Your Own Video Discovery Projects
- Separate pipelines for content understanding and external signals; this decouples heavy video processing from fastâmoving trend data.
- Leverage foundation models (via Bedrock) for scene reasoning and captioning, but keep the models behind your own guardrails.
- Define editorial actions that map directly to marketer intent; avoid forcing users to type ambiguous queries.
- Use serverless OpenSearch for flexible indexing of both vector embeddings and traditional metadata.
- Implement human review as the final gateâAI can propose, but marketers must decide.
---
Closing Thoughts
Social Departmentâs QuickSearch demonstrates that cultureâaware video discovery is achievable when you combine robust AWS services with clear editorial intent. By turning subjective marketing questions into concrete retrieval actions, the system delivers the right clip at the right momentâwithout requiring marketers to watch hours of footage first. The pattern of independent content and trend pipelines, Bedrockâpowered reasoning, and responsibleâAI safeguards can serve as a blueprint for any organization looking to make large video libraries searchable in a way that truly reflects audience culture.
Source
Read original sourceWhy we picked this
Meaningful AI news showcasing a video discovery use case built with Bedrock models.