Developing QuantWare
QuantWare – Building an Uncensored Transformer-Based LLM
Our model, QuantWare, is built on the Transformer architecture, which underpins virtually all modern open and closed Large Language Models (LLMs). This architecture enables the model’s core functionality: predicting the next token based on input sequences, or, in other words, generating natural language outputs that are coherent and understandable to humans.
QuantWare provides multiple checkpoints tailored for various applications, from general-purpose language modeling to specialized chat and instruction-following tasks. Its versatility makes it ideal for use cases such as chatbots, content generation, and solving complex problems across multiple domains.
We strongly believe that an LLM can only achieve its full potential when free from censorship. Evidence from recent studies at Stanford University and UC Berkeley suggests that models like GPT-3.5 and GPT-4 show diminished performance due to the implementation of extensive guardrails and hardcoded biases, which inherently limit their capabilities.
Training QuantWare
The first stage in building QuantWare involved data collection and preparation, incorporating a diverse range of sources: books, articles, scientific papers, audio transcriptions, publicly available data, and datasets provided by contributors. While primarily focused on English, the dataset spans multiple languages. At a scale of trillions of tokens, the data was carefully preprocessed and segmented to match the requirements of our Transformer architecture.
Once the dataset was ready, we defined hyperparameters—including the number of layers, hidden unit sizes, and attention heads—and initiated unsupervised pre-training. This process was conducted in a distributed training protocol over multiple iterations until the desired performance targets were achieved.
Making QuantWare Truly Uncensored
Achieving an uncensored LLM involves addressing two major challenges: training bias and hardcoded output bias.
Training Bias
LLMs, like humans, are influenced by the data they are exposed to. If a human sees only a single perspective, their understanding is inherently biased. Similarly, an LLM trained on a limited dataset will reflect those biases in its billions of parameters, generating outputs influenced by that initial data. To counteract this, we expanded our dataset to cover knowledge gaps left by other models, ensuring broader coverage and more balanced perspectives.
We also acknowledge that knowledge is constantly evolving. To remain relevant, QuantWare will undergo periodic retraining to incorporate new information, including recent articles, books, scientific papers, copyright-free material, and proprietary datasets contributed by our community. Updated versions of QuantWare will be released regularly, informed both by new data and user feedback.
Hardcoded Output Bias
Hardcoded biases occur when outputs are altered or filtered before delivery to the user, analogous to a censoring department reviewing content before publication. These biases do not originate from the LLM itself but are imposed externally. Our approach is simple and transparent: we do not implement any hardcoded biases, ensuring that QuantWare delivers uncensored, unfiltered outputs directly from the model.
// QUANTWARE //Last updated