Consoles is a simple and lightweight framework for developing backends and interacting with LLMs, giving them access to structured prompting, code execution, and other tools. It offers a seamless interface for interacting with various AI providers, including OpenAI, Anthropic, and Cloudflare, ensuring efficient resource management and cost calculation.
- Multi provider & model support: Seamlessly switch between various models from providers such as OpenAI, Anthropic, Cloudflare and Google using the integrated framework.
- Granular Configuration: Customize each call with options like provider, model, temperature, max tokens, etc. This allows you to adapt each interaction to specific needs, giving you full control over your models' behavior and performance.
- Structured JSON Responses: Automatically handle and ensure responses are in valid JSON format when required. This feature ensures that responses are formatted as valid JSON objects, even when using models that do not have built-in JSON support. (Note: Performance may degrade with lesser model types.)
- User-friendly Prompting: Utilize intuitive and context-aware prompting directly within functions, making it easy to generate and handle prompts natively within the framework.
- Schema-based Response Validation: Define and enforce response types using Zod schemas, ensuring that all responses adhere to the expected content, structure, and data types.
- Advanced Error Handling: Implement robust error handling and logging mechanisms for enhanced debugging and reliability.
To get started with ConsoleAI, follow these steps:
-
Install the
consoles-aipackage using npm:npm install consoles-ai
-
Import the
LLMclass from theconsoles-aipackage in your project:import { LLM } from 'consoles-ai'; import { z } from 'zod';
-
Configure the
LLMinstance in your code.// Start a new LLM instance const llm = new LLM(); llm.config({ provider: 'openai', keys: { openai: 'your-openai-key', }, model: 'gpt-4o', temperature: 0.5, }); // Define a simple schema for a greeting const greetingSchema = z.object({ greeting: z.string().describe('A simple greeting message') }); // Function to generate a simple greeting response async function generateGreetingResponse(greeting) { const prompt = { system: `You are a friendly assistant generating a simple greeting response.`, user: `Generate a response for the greeting: "${greeting}".`, schema: greetingSchema }; return await llm.chat(prompt, { model: 'gpt-3.5-turbo', temperature: 0.7, maxTokens: 50 }); } // Example usage of the new functions const userGreeting = 'Hi there!' generateGreetingResponse(userGreeting).then(response => { console.log('Response:', response); });
- Node.js 14+ (TypeScript required)
The consolesai package supports a variety of providers and models for Large Language Models (LLMs). Below is a list of the currently supported providers and their respective models:
gpt-4ogpt-4-turbogpt-4gpt-3.5-turbogpt-4o-2024-05-13gpt-4-turbo-2024-04-09gpt-4-turbo-previewgpt-4-0125-previewgpt-4-1106-previewgpt-4-vision-previewgpt-4-1106-vision-previewgpt-4-0613gpt-4-32kgpt-4-32k-0613gpt-3.5-turbo-0125gpt-3.5-turbo-1106gpt-3.5-turbo-instruct
-
claude-3-opus-20240229 -
claude-3-sonnet-20240229 -
claude-3-haiku-20240307 -
claude-3-opus-20240229 -
claude-3-sonnet-20240229 -
claude-3-haiku-20240307
@cf/meta/llama-2-7b-chat-fp16@cf/mistral/mistral-7b-instruct-v0.1@cf/thebloke/deepseek-coder-6.7b-base-awq@cf/thebloke/deepseek-coder-6.7b-instruct-awq@cf/deepseek-ai/deepseek-math-7b-base@cf/deepseek-ai/deepseek-math-7b-instruct@cf/thebloke/discolm-german-7b-v1-awq@cf/tiiuae/falcon-7b-instruct@cf/google/gemma-2b-it-lora@cf/google/gemma-7b-it@cf/google/gemma-7b-it-lora@hf/nousresearch/hermes-2-pro-mistral-7b@hf/thebloke/llama-2-13b-chat-awq@cf/meta-llama/llama-2-7b-chat-hf-lora@cf/meta/llama-3-8b-instruct@cf/meta/llama-3-8b-instruct-awq@hf/thebloke/llamaguard-7b-awq@hf/thebloke/mistral-7b-instruct-v0.1-awq@hf/mistral/mistral-7b-instruct-v0.2@cf/mistral/mistral-7b-instruct-v0.2-lora@hf/thebloke/neural-chat-7b-v3-1-awq@cf/openchat/openchat-3.5-0106@openhermes-2.5-mistral-7b-awq@cf/microsoft/phi-2@cf/qwen/qwen1.5-0.5b-chat@cf/qwen/qwen1.5-1.8b-chat@cf/qwen/qwen1.5-14b-chat-awq@cf/qwen/qwen1.5-7b-chat-awq@cf/defog/sqlcoder-7b-2@hf/nexusflow/starling-lm-7b-beta@cf/tinyllama/tinyllama-1.1b-chat-v1.0@cf/fblgit/una-cybertron-7b-v2-bf16@hf/thebloke/zephyr-7b-beta-awq
- (Currently no models listed)
- (Currently no models listed)
This list is subject to change as new models and providers are added to the consolesai package. Always refer to the latest documentation for the most up-to-date information.
This tool provides a set of commands to help you manage and deploy your Consoles AI projects with ease.
Install the Consoles AI CLI globally using npm:
npm install -g consoles-aiinit: Initialize a new Consoles AI projectsetup: Set up your Consoles AI environmentdeploy: Deploy your project to the Consoles AI cloudrun: Execute a script in the Consoles AI cloud environmentkv: Manage your Key-Value storesecrets: Manage your Secret Spaces for secure environment variableshelp: Display help informationversion: Check the current CLI version
consoles <command> [options]consoles init
consoles setup
consoles deploy [entry_script]
consoles run <script>
consoles kv [action] [namespace] [key] [value]
consoles secrets [action] [space] [key] [value]
consoles help
consoles -v- Install the CLI globally using npm
- Run
consoles initto create a new project - Use
consoles setupto configure your environment - Develop your project using Consoles AI features
- When ready, use
consoles deployto push your project to the cloud
For more information and detailed documentation, visit our website or reach out to our support team.
Happy coding with Consoles AI! 🚀✨