What You Need to Know About Musk's Grok

Musk's AI company, announced their new product Grok, which is designed to compete with OpenAI's ChatGPT and has aroused curiosity among netizens. Unlike ChatGPT, Grok comes with a sense of humor and sarcasm when answering questions.

When asked dangerous questions such as "how to make cocaine," Grok first pretends to give serious answers with some joking-like suggestions, and then tells you in a relaxed tone that it is illegal.

In another screenshot, Grok's sense of humor is even more evident. It was asked to retrieve the latest news about "SBF" from the past day (note: SBF, Sam Bankman-Fried's full name, is a prominent figure in the cryptocurrency industry), and it gave a sarcastic response with its answer.

In addition to these screenshots, xAI also revealed the information behind Grok's large model through a blog post. It shows that Grok-1, the large model behind Grok, achieved a good level after only two months of training, but did not surpass GPT-4, and its supported context length was not long. During the training, Grok-1 used Jax, a deep learning framework, instead of PyTorch.

xAI said that Grok's dialogue ability is currently supported by a large model called Grok-1, which they developed over the past four months and went through multiple iterations. After announcing the establishment of xAI, they trained a 33 billion parameter LLM prototype - Grok-0. This early model approached the ability of LLaMA 2 (70B) on standard LM benchmark tests, but only used half of the training resources. In the past two months, they have made significant improvements to the model's reasoning and encoding capabilities and finally developed Grok-1, a more powerful SOTA language model that achieved a score of 63.2% in HumanEval coding tasks and 73% in MMLU.

In these benchmark tests, Grok-1 demonstrated strong performance, surpassing all other models in its computation class, including ChatGPT-3.5 and Inflection-1. Only models trained with a large amount of training data and computing resources like GPT-4 can exceed it. xAI said that this demonstrates their rapid progress in efficient training of LLMs.