Amazon SageMaker

This document will guide you through enabling development teams to access Cohere’s offerings on Amazon SageMaker.

Prerequisites

In order to successfully subscribe to Cohere’s offerings on SageMaker, the user will need the following Identity and Access Management (IAM) permissions:

AmazonSageMakerFullAccess
aws-marketplace:ViewSubscriptions
aws-marketplace:Subscribe
aws-marketplace:Unsubscribe

These permissions allow a user to manage your organization’s SageMaker subscriptions. Learn more about managing Amazon’s IAM Permissions here. Contact your AWS administrator if you have questions about account permissions.

Cohere with Amazon SageMaker Setup

First, navigate to Cohere’s Sagemaker Marketplace to view the product offerings available to you. Select the product offering to which you are interested in subscribing.

Next, explore the tools on the Product Detail page to evaluate how you want to configure your subscription. Some of the key sections to consider are detailed below:

Pricing

This section allows you to estimate the cost of running inference on different types of instances.

(a) Version: Select the model version that suits your use case. Typically selecting the most recent version is appropriate. Release notes can be found in the Overview section of the listing.

(b) Region: Select a region to see which instance types are available.

(c) Instance Type: Select the instance type to estimate the costs of different configurations (learn more about Amazon instance types here). Typically, instances that cost more will have lower latencies, meaning they will produce generations faster. Model quality is unaffected by instance type. Cohere provides a recommended instance type for each of our product offerings.

Note: Due to regional availability and high demand, some supported instance types may not be available from Amazon at all times. Cohere does not control which instance types are available on Amazon’s platform.

Select version (a), region (b), and instance type (c) to get pricing estimates

Usage Information

This section contains the technical details around supported data formats for each model, and offers links to documentation and notebooks that will help developers scope out the effort required to integrate with Cohere’s models.

View sample inputs and outputs, accepted data formats and other developer resources

Each product offering has a corresponding notebook in GitHub that guides developers through the process of creating an endpoint for the model and performing real-time inference against it.

List of notebooks that offer guidance on deploying Cohere’s products via SageMaker

Notebooks for all of Cohere’s SageMaker compatible models are found here.

Subscribing

When you are ready to subscribe to one of Cohere’s models, select the Continue to Subscribe button at the top of the Product Details page.

On the Subscribe page, you are presented again with the pricing details and the EULA for final review before accepting the offer. This information is identical to the information on Product Detail page. If the product includes a Free Trial, that will be indicated on this page. Selecting different options in the Version will update the pricing displayed on the page, but does not impact your implementation in any way.

When you are ready to proceed with subscription, select Accept Offer. This request may take a few seconds to complete.

Note: Subscribing to the models alone will not result in charges to your account, this only enables your organization to proceed with setting up SageMaker instances with Cohere’s models. You will begin accruing charges when you create a SageMaker instance configured to run one of Cohere's products.

Select “Accept Offer” when you are ready to proceed with your subscription

Configuration

When Amazon has processed your subscription, select Continue to configuration.

On the Configure page, the primary goal is to retrieve the Amazon Resource Name (ARN) for the product you have subscribed to. To do this, first select both a) Software Version and b) Region, at which point the Product ARN will be displayed, which you can copy via the c) copy icon.

Select Software Version (a) and Region (b) to surface Product ARN (c)

With your selected configuration and Product ARN available, you now have everything you need to integrate with Cohere’s model offerings on SageMaker.

Cohere recommends your next step be to find the appropriate notebook in Cohere's list of SageMaker notebooks, and follow the instructions there, or provide the link to Cohere’s SageMaker notebooks to your development team to implement. The notebooks are thorough, developer-centric guides that will enable your team to begin leveraging Cohere’s endpoints in production for live inference.

If you have further questions about subscribing or configuring Cohere’s product offerings on Amazon SageMaker, please contact our team at [email protected].