Researchers from MIT and several European institutions have developed a new technique called CompreSSM that makes artificial intelligence models leaner and faster during the learning phase itself, according to a recent announcement.
The new method integrates compression directly into the AI training process, avoiding the typical post-training model shrinking that can degrade accuracy. This approach enables powerful AI systems with significantly lower computational and energy costs, streamlining the creation of sophisticated models for applications from natural language processing to robotics.
What We Know So Far
- A team of researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL), the Max Planck Institute, the European Laboratory for Learning and Intelligent Systems, ETH, and Liquid AI developed the new method, named CompreSSM, as reported by MIT News.
- The technique compresses AI models during the training process, rather than as a separate step afterward, which helps avoid the typical performance loss associated with post-training compression.
- According to the researchers, CompreSSM uses mathematical tools from control theory to identify and remove redundant or unimportant components from the model early in its training.
- The process is effective because the relative importance of a model's different components tends to stabilize after just 10 percent of the total training process has been completed, the MIT report stated.
