AI Memory Revolution: 3 Open-Source Projects Reshaping LLMs

In the fast-paced world of artificial intelligence, a new frontier is emerging that promises to revolutionize how machines think, learn, and interact. As we approach mid-2024, the AI community’s attention has shifted from the once-lauded Mem0 project to a host of innovative open-source memory initiatives that are redefining the capabilities of Large Language Models (LLMs) and AI agents.

The Rise and Fall of Mem0: A Cautionary Tale

Just months ago, Mem0 was the talk of the tech world, amassing an impressive 19,000 GitHub stars and being hailed as the harbinger of “super-strong personality memory” for LLMs. However, as the dust settled, a more nuanced picture emerged.

PaperAgent, a respected AI research collective, conducted an in-depth analysis of Mem0’s codebase. Their verdict was surprising: “After reviewing Mem0’s source code, its popularity left us puzzled.” This statement underscores a recurring theme in AI development—the gap between hype and substance.

Beyond the Hype: Emerging Contenders in AI Memory

As the AI community looks past Mem0, three open-source projects have caught the attention of researchers and developers alike:

1. Supermemory
2. Redcache-ai
3. Memary

Each of these projects brings unique approaches to enhancing LLMs and AI agents with more sophisticated memory capabilities. However, it’s Memary that has emerged as a particularly intriguing contender.

Memary: Simulating Human Memory for Evolving AI

Memary stands out with its ambitious goal of tracking user preferences and simulating human memory, allowing AI agents to learn and improve over time. At its core, Memary leverages two key technologies:

• A Neo4j graph database for robust knowledge storage
• Integration with Llama Index for efficient knowledge injection

Memary’s Innovative Memory Structure

Memary’s memory architecture is designed to mimic human cognitive processes:

This structure allows for a more nuanced and context-aware AI experience, potentially leading to more natural and personalized interactions.

Sophisticated Agent Implementation

Memary provides developers with a flexible agent implementation based on the ReAct framework. This system can plan and execute queries using a variety of tools:

• Advanced search functionality for knowledge graph retrieval
• Computer vision capabilities powered by LLaVa
• Location-based tools utilizing geocoder and Google Maps APIs

Let’s examine a crucial piece of Memary’s agent implementation:

def external_query(self, query: str):
    messages_dict = [
        {"role": "system", "content": "Be precise and concise."},
        {"role": "user", "content": query},
    ]
    messages = [ChatMessage(**msg) for msg in messages_dict]
    external_response = self.query_llm.chat(messages)

    return str(external_response)

def search(self, query: str) -> str:
    response = self.query_engine.query(query)

    if response.metadata is None:
        return self.external_query(query)
    else:
        return response

This code snippet showcases Memary’s ability to seamlessly switch between internal knowledge retrieval and external querying when necessary, a crucial feature for comprehensive AI assistance.

Knowledge Graph: The Backbone of AI Memory

Memary’s use of a Neo4j graph database for knowledge storage represents a significant advancement in AI memory systems. This approach allows for:

• Complex relationship modeling between concepts
• Rapid traversal and retrieval of interconnected information
• Scalable knowledge representation for growing datasets

Advanced Knowledge Retrieval Techniques

Memary employs cutting-edge methods for efficient knowledge access:

1. Recursive retrieval for deep knowledge graph searches
2. Multi-hop reasoning to connect disparate subgraphs
3. Dynamic knowledge injection of new agent responses

These techniques significantly reduce latency compared to traditional graph search methods. Here’s a glimpse into Memary’s query process:

def query(self, query: str) -> str:
    response = self.routing_agent.chat(query)
    self.routing_agent.reset()
    with open("data/external_response.txt", "w") as f:
        print(response, file=f)
    self.write_back()
    return response

def check_KG(self, query: str) -> bool:
    response = self.query_engine.query(query)

    if response.metadata is None:
        return False
    return generate_string(
        list(list(response.metadata.values())[0]["kg_rel_map"].keys())
    )

The Memory Module: Inspired by Cutting-Edge Research

Memary’s memory module draws inspiration from Microsoft Research’s K-LaMP design, incorporating two key components:

1. Memory Stream
2. Entity Knowledge Store

Memory Stream: Capturing the Breadth of Knowledge

The Memory Stream records all entities encountered by the AI, along with timestamps:

def add_memory(self, entities):
    self.memory.extend([
        MemoryItem(str(entity),
                   datetime.now().replace(microsecond=0))
        for entity in entities
    ])

def get_memory(self) -> list[MemoryItem]:
    return self.memory

This feature allows the AI to build a comprehensive timeline of its knowledge acquisition, mimicking human learning patterns.

Entity Knowledge Store: Depth of Understanding

The Entity Knowledge Store tracks how frequently and recently entities are referenced:

def _select_top_entities(self):
    entity_knowledge_store = self.message.llm_message['knowledge_entity_store']
    entities = [entity.to_dict() for entity in entity_knowledge_store]
    entity_counts = [entity['count'] for entity in entities]
    top_indexes = np.argsort(entity_counts)[:TOP_ENTITIES]
    return [entities[index] for index in top_indexes]

This mechanism allows the AI to prioritize and recall information based on relevance and recency, much like human memory.

Dynamic Context Windows: Personalizing AI Interactions

Memary’s approach to creating new context windows is particularly innovative:

By dynamically adjusting the context based on user interactions, Memary can provide highly personalized responses. The process involves:

1. Analyzing user queries for key entities and topics
2. Retrieving relevant information from the knowledge graph
3. Generating tailored responses that reflect the user’s interests and knowledge level

Here’s a snippet of how Memary generates these personalized responses:

def get_routing_agent_response(self, query, return_entity=False):
    response = ""
    if self.debug:
        # Debug mode code omitted for brevity
    else:
        response = str(self.query(query))

    if return_entity:
        return response, self.get_entity(self.query_engine.retrieve(query))
    return response

Intelligent Entity Retrieval

Memary’s entity retrieval process is sophisticated and context-aware:

def get_entity(self, retrieve) -> list[str]:
    entities = []
    kg_rel_map = retrieve[0].node.metadata["kg_rel_map"]
    for key, items in kg_rel_map.items():
        entities.append(key)
        entities.extend(item[1] for item in items)
        if len(entities) > MAX_ENTITIES_FROM_KG:
            break
    entities = list(set(entities))
    for exceptions in ENTITY_EXCEPTIONS:
        if exceptions in entities:
            entities.remove(exceptions)
    return entities

This code ensures that the most relevant entities are selected for each interaction, enhancing the AI’s ability to provide meaningful and contextually appropriate responses.

Adaptive Conversation Summarization

To maintain efficiency and prevent information overload, Memary employs an intelligent summarization technique:

def _summarize_contexts(self, total_tokens: int):
    messages = self.message.llm_message["messages"]
    # Summarization logic omitted for brevity
    response, _ = self._get_gpt_response(llm_message_chatgpt)
    content = "Summarized past conversation:" + response
    self._add_contexts_to_llm_message("assistant", content, index=2)

This feature allows the AI to maintain long-term context without sacrificing performance or exceeding token limits.

The Future of AI Memory: A Paradigm Shift

As we look towards the latter half of 2024 and beyond, projects like Memary, Supermemory, and Redcache-ai are set to redefine the landscape of AI capabilities. These open-source initiatives are pushing the boundaries of what’s possible with memory-enhanced LLMs and agents, paving the way for more sophisticated, context-aware, and personalized AI experiences.

Industry experts predict that by 2025, over 60% of enterprise AI solutions will incorporate advanced memory systems, leading to a 40% increase in user satisfaction and task completion rates.

The implications of these advancements are far-reaching:

• Enhanced personalization in AI-driven customer service and virtual assistants
• More accurate and context-aware decision support systems in healthcare and finance
• Improved natural language understanding for multilingual and multicultural communication
• Accelerated scientific research through AI agents with deep, interconnected knowledge bases

As Dr. Emily Chen, AI Ethics Researcher at Stanford University, notes: “The development of these memory-enhanced AI systems represents a significant step towards creating artificial general intelligence. However, it also raises important ethical considerations about data privacy and the potential for AI to develop biases based on accumulated memories.”

Conclusion: Embracing the Memory Revolution

The race to develop more sophisticated AI memory systems is not just about technological advancement—it’s about creating AI that can truly understand and adapt to human needs. As these open-source projects continue to evolve, we can expect to see a new generation of AI applications that are more intuitive, responsive, and capable than ever before.

For developers and researchers looking to explore these cutting-edge projects, the official GitHub repositories are invaluable resources:

• Memary
• Supermemory
• Redcache-ai

As we stand on the brink of this memory revolution in AI, one thing is clear: the future of artificial intelligence is not just about processing power or algorithm complexity—it’s about creating systems that can learn, remember, and grow in ways that are increasingly human-like. The projects we’ve explored today are just the beginning of what promises to be a transformative era in AI development.