Cohere Problems & Issues
Real user-reported problems, complaints, and limitations for Cohere. Based on analysis of 15 signals.
Issues by Category
Not a plug-and-play solution - heavy engineering required
Cohere provides powerful AI models and APIs, but users need to build the entire application including UI and integrations. Requires working with APIs, building interfaces, and managing the whole setup. A huge barrier for teams without spare engineering resources.
Heavy sales process before access
Getting started involves a chain of sales calls and mandatory demos before teams can even touch the product. This is followed by a major API integration project with engineering teams. Not suitable for organizations wanting quick evaluation.
Fine-tuning requires deep ML expertise
Fine-tuning could be simplified to support broader teams without deep ML expertise. Currently, the process is complex and requires specialized knowledge that many enterprise teams don't have in-house.
Enterprise pricing completely opaque
For main enterprise platforms like North and Compass, pricing is opaque requiring direct sales contact for custom quotes. Annual costs can range from free API tier up to $100,000+ for enterprise plans. Budgeting is difficult without going through sales.
Per-token pricing makes budgeting unpredictable
Pricing models based on 'per token' or 'per resolution' charges can make budgeting unpredictable for finance teams. Usage can spike unexpectedly, and cost projections are difficult when input sizes vary significantly across use cases.
Rate limiting on trial accounts frustrates evaluation
Rate limiting on trial accounts is noted as a pain point. Teams trying to evaluate Cohere for enterprise use cases hit limits that prevent meaningful testing, making it difficult to assess fitness before committing.
Command model lacks creativity for open-ended tasks
The Command model is not as creative as larger LLMs available in the market. This limitation is noticeable in open-ended generative tasks. For creative writing or ideation, competitors like GPT-4 and Claude significantly outperform.
Embedding distances much higher than OpenAI
Compared to OpenAI embedding models, Cohere's distances are much higher (around 0.5 or more) versus OpenAI's 0.005. This can require extra processing steps and affects similarity search workflows that were calibrated for OpenAI embeddings.
Dashboard analytics lack detail
Reporting and analytics in the dashboard could be more detailed and fine-tuned. For teams needing deep insights into usage patterns and performance metrics, the current analytics are insufficient.
Context window doesn't work as advertised
Despite claiming a 128k context window, the Command R+ model begins producing nonsensical output after 8192 tokens. This significant gap between advertised and actual functionality affects document-heavy workflows.
Knowledge cutoff creates accuracy issues
Models were trained with data only through February 2023. Questions requiring information from after this date will likely produce incorrect answers. For current events or recent developments, the model is unreliable.
Copyright lawsuits create legal uncertainty
Major publishers including The Atlantic, Condé Nast, Forbes, and Toronto Star sued Cohere for scraping articles to train LLMs. A judge rejected Cohere's motion to dismiss. The ongoing litigation creates legal uncertainty for enterprise customers.
Safety issues with long conversations
Despite safeguards being in place, it is still possible to encounter toxicity especially over long conversations with multiple turns. Cohere does not recommend using Command R models alone for decisions that could significantly impact individuals.
Models have fallen behind state-of-the-art
TechCrunch noted that Cohere's models 'have fallen behind state-of-the-art' in raw performance. While focused on enterprise RAG use cases, the models don't match GPT-4o or Claude on general benchmarks. Slower growth than leading rivals.
May require abandoning existing helpdesk systems
Some Cohere AI solutions require users to completely change how they work, potentially requiring migration away from helpdesk systems they've used for years like Zendesk or Intercom. This disrupts established workflows and increases switching costs.
Regret Patterns
Common reasons users regret choosing Cohere.
Underestimated engineering requirements
Teams expected a simpler implementation but discovered Cohere requires building entire applications around APIs. Without dedicated AI engineering resources, projects stalled or failed. The plug-and-play solution they expected doesn't exist.
Spent months in sales process before evaluation
Organizations went through extended sales cycles with mandatory demos before even touching the product. By the time they could evaluate it, significant time had passed. Some discovered it didn't fit their needs after all.
Hit context window limits in production
After building document processing workflows on the advertised 128K context window, teams discovered output degrades after 8K tokens. Production systems failed on large documents that should have worked. Significant rework required.
Legal raised concerns about copyright lawsuits
Enterprise legal teams flagged the ongoing publisher lawsuits during procurement review. Some organizations had to pause or abandon Cohere implementations until legal uncertainty is resolved. Wasted evaluation effort.
Realized models fell behind during implementation
Teams committed to Cohere based on earlier benchmarks, then discovered during implementation that competitors had surged ahead. The models that seemed competitive at evaluation were outdated by deployment. Switching costs made migration painful.
Discovered enterprise costs after deep integration
Free tier rate limits forced enterprise discussions late in evaluation. Custom pricing revealed costs far exceeding initial estimates. By this point, significant engineering time was invested, creating lock-in pressure.
When Cohere Breaks Down
Specific scenarios where Cohere fails or underperforms.
Document exceeds actual context window capacity
Despite 128K advertised context, output becomes nonsensical after 8192 tokens. Large documents that should process correctly produce garbage output. Document processing workflows fail on enterprise-scale content.
Team lacks engineering resources for integration
Without dedicated AI engineers, implementation stalls. The APIs are powerful but require significant development work. Teams without ML expertise can't complete fine-tuning or optimize deployment.
Creative use case requires open-ended generation
When workflows require creative content generation, Command models underperform. Marketing, copywriting, and ideation tasks produce mediocre results compared to GPT-4 or Claude. Need to supplement with competitor models.
Legal compliance review flags copyright issues
Enterprise procurement processes surface the ongoing publisher lawsuits. Legal teams may require risk assessments or waiting for resolution. Can halt implementations during review.
Usage scales beyond free tier limits
Evaluation hits rate limits preventing meaningful testing. Transitioning to paid requires sales discussions and potentially enterprise commitments. Budget unpredictability with per-token pricing creates finance concerns.
Migration from existing helpdesk required
Some implementations require abandoning Zendesk, Intercom, or other established tools. The workflow disruption and retraining costs exceed the AI benefits. Teams abandon implementation to preserve existing processes.