Cohere Problems & Issues

Cohere provides powerful AI models and APIs, but users need to build the entire application including UI and integrations. Requires working with APIs, building interfaces, and managing the whole setup. A huge barrier for teams without spare engineering resources.

Getting started involves a chain of sales calls and mandatory demos before teams can even touch the product. This is followed by a major API integration project with engineering teams. Not suitable for organizations wanting quick evaluation.

Fine-tuning could be simplified to support broader teams without deep ML expertise. Currently, the process is complex and requires specialized knowledge that many enterprise teams don't have in-house.

TechCrunch noted that Cohere's models 'have fallen behind state-of-the-art' in raw performance. While focused on enterprise RAG use cases, the models don't match GPT-4o or Claude on general benchmarks. Slower growth than leading rivals.

For main enterprise platforms like North and Compass, pricing is opaque requiring direct sales contact for custom quotes. Annual costs can range from free API tier up to $100,000+ for enterprise plans. Budgeting is difficult without going through sales.

Pricing models based on 'per token' or 'per resolution' charges can make budgeting unpredictable for finance teams. Usage can spike unexpectedly, and cost projections are difficult when input sizes vary significantly across use cases.

Rate limiting on trial accounts is noted as a pain point. Teams trying to evaluate Cohere for enterprise use cases hit limits that prevent meaningful testing, making it difficult to assess fitness before committing.

The Command model is not as creative as larger LLMs available in the market. This limitation is noticeable in open-ended generative tasks. For creative writing or ideation, competitors like GPT-4 and Claude significantly outperform.

Compared to OpenAI embedding models, Cohere's distances are much higher (around 0.5 or more) versus OpenAI's 0.005. This can require extra processing steps and affects similarity search workflows that were calibrated for OpenAI embeddings.

Reporting and analytics in the dashboard could be more detailed and fine-tuned. For teams needing deep insights into usage patterns and performance metrics, the current analytics are insufficient.

Some Cohere AI solutions require users to completely change how they work, potentially requiring migration away from helpdesk systems they've used for years like Zendesk or Intercom. This disrupts established workflows and increases switching costs.

Despite claiming a 128k context window, the Command R+ model begins producing nonsensical output after 8192 tokens. This significant gap between advertised and actual functionality affects document-heavy workflows.

Models were trained with data only through February 2023. Questions requiring information from after this date will likely produce incorrect answers. For current events or recent developments, the model is unreliable.

Major publishers including The Atlantic, Condé Nast, Forbes, and Toronto Star sued Cohere for scraping articles to train LLMs. A judge rejected Cohere's motion to dismiss. The ongoing litigation creates legal uncertainty for enterprise customers.

Despite safeguards being in place, it is still possible to encounter toxicity especially over long conversations with multiple turns. Cohere does not recommend using Command R models alone for decisions that could significantly impact individuals.