π Runtime, LLM-powered NPCs
Interactive_NPCs.mp4
Features
- πΉοΈ NPCs that
<speak>,<jump>,<attack>and perform any other action you've defined - β‘ <1 second GPU inference on most machines
- π€ Open-weights models available, fined-tuned from: Llama-3, Phi-3, Mistral, etc.
- π Structured generation with Outlines γ°οΈ means the output format is always respected
- ποΈ Coming soon: Local server mode, with language-agnostic API
- π Available on API: Runtime quest generation, for players and NPCs
- πΆβπ«οΈ Available on API: Memory creation, storage and retrieval with a Vector DB
Gigax has new releases and features on the way. Make sure to β star and π watch this repository!
Usage
Model instantiation
-
We provide various models on the π€ Huggingface hub:
- NPC-LLM-7B (our Mistral-7B fine-tune)
- NPC-LLM-3_8B (our Phi-3 fine-tune)
- NPC-LLM-3_8B-128k (our Phi-3 128k context length fine-tune)
-
All these models are also available in gguf format to run them on CPU using llama_cpp
-
Start by instantiating one of them using outlines:
from outlines import models from gigax.step import NPCStepper from transformers import AutoTokenizer, AutoModelForCausalLM # Download model from the Hub llm = Llama.from_pretrained( repo_id="Gigax/NPC-LLM-3_8B-GGUF", filename="npc-llm-3_8B.gguf" # n_gpu_layers=-1, # Uncomment to use GPU acceleration # n_ctx=2048, # Uncomment to increase the context window ) model = models.LlamaCpp(llm) # Instantiate a stepper: handles prompting + output parsing stepper = NPCStepper(model=model)
Stepping an NPC
- From there, stepping an NPC is a one-liner:
action = await stepper.get_action( context=context, locations=locations, NPCs=NPCs, protagonist=protagonist, items=items, events=events, )
- We provide classes to instantiate
Locations,NPCs, etc. :
from gigax.parse import CharacterAction from gigax.scene import ( Character, Item, Location, ProtagonistCharacter, Skill, ParameterType, ) # Use sample data context = "Medieval world" current_location = Location(name="Old Town", description="A quiet and peaceful town.") locations = [current_location] # you can add more locations to the scene NPCs = [ Character( name="John the Brave", description="A fearless warrior", current_location=current_location, ) ] protagonist = ProtagonistCharacter( name="Aldren", description="Brave and curious", current_location=current_location, memories=["Saved the village", "Lost a friend"], quests=["Find the ancient artifact", "Defeat the evil warlock"], skills=[ Skill( name="Attack", description="Deliver a powerful blow", parameter_types=[ParameterType.character], ) ], psychological_profile="Determined and compassionate", ) items = [Item(name="Sword", description="A sharp blade")] events = [ CharacterAction( command="Say", protagonist=protagonist, parameters=[items[0], "What a fine sword!"], ) ]
API
Contact us to give our NPC API a try - we'll take care of model serving, NPC memory, and more!