It's the first time in the history of the world.
A business card with 96GB of storage$13.8 millionA 128GB DDR5 memory set is approaching$250,000。
Even Cook, who has been silent ever since, has described this round of memory storms in the financial statements."One Hundred Years of Flooding"。
It's a disgrace.The more expensive the hardware, the more AI companies go out of business.OpenAI has bought tens of thousands of Macs to train agents to run their own computers.
The reason for this is simple: A 24-hour axis-spinning Agent can burn the bills if all the chips are running in the cloud, and a large model is already moving to the edge.
But to buy back the chips and memory, only half a ticket was paid.
To make a large model into a stable, useful end product, model adaptation, framework opening, performance optimization, the development team doesn't have to start from scratch if there are already tools and experience.
At today's Ecological Partners Conference, PMO Intel announced that the company will continue to expand its business in the coming years.The whole AI ecosystem software kit will be fully open source!
From getting the model to making the product, the entire tool chain is open source.By January next year,15 open source projectsIt'll be up and running.
The M50 chip has been mass-produced for less than a year.More than 150 terminal products set out to use it, hundreds of collaborative customers, and even put the 122-billion-parameter model in a 300-g small box.
The software package is designed to be developed, verified and refined in these projects.
An entire AI landing line, fully open source
To be an offline conference assistant, it looks like three steps: recording, transcribing, writing the memo, and finally storing it on the computer.
It's really hard to do, but the problem arises:The model is fortressed in a limited memory, and the reasoning is that the chip, voice recognition, and the large model must work in sync.It's not easy to run, and there's performance optimization, stability testing and line maintenance behind it.
What's even more troubling is that when the next team makes the same products, they often have to do it all over again.
This open-source tool chain covers the entire process, from getting the model to the product shelf, in four layers.
The first layer is model processing tools.
The developers got a new model and used it first.Quantify, compress, compile and accurately evaluate modelsIt's the only way to make it a chip-friendly format.
Typically, models are embedded in edge devices, and often have to be hard-pressed into a low-precision format to get rid of the precision, and engineers spend a lot of time optimizing to get back to fixing.
And what?M50 natively supports floating point computingDevelopers can fully adapt to their needs.Directly deployed in floating point format, reducing one layer of folding from the source.
The second layer is the reasoning framework, the arithmetic library, and the running time.
This layer is equivalent toTotal adjustment on the chipIt is responsible for delivering computational task accuracy and optimizing the storage architecture for the M50.
The model is running fast and slow, taking up as much memory and power as it can squeeze, and that's the whole layer of scheduling power.
The third layer is field kits, model flow frameworks and intermediates.
It is responsible.Let's work together like a symphony orchestraThe conference assistant, voice transcripts, text transcripts, memos are stored in the application, and every interaction between them is smoothed out using this layer.
The fourth layer is the deployment of transportation tools.
After the delivery of the product, the subsequent upgrades, monitoring and management of large batches of equipment are entirely under its control.
These hard-core tools are being prepared in three open-source batches.
The first is an adaptation of llama.cpp, vLLM.These are the most commonly used open-source reasoning frameworks in the developer community.It's not necessary to change your habits and use the best possible workflow to access the chip.
The second is a self-developed frame of reasoning.In addition to this, it is possible to improve the performance of its own chips by making deep optimizations.
The third group is intermediaries and tools.It's the ability to bulk up a bunch of models.It's a product that sells money.
In the entire open source list,ModelZoo (model example library), reasoning engines, field kits must be complete.The Kubernetes infrastructure for managing large amounts of accelerated cards and the NPU card pool management platform have also been integrated.
Now go to GitHub to search for houmo-ai and you can see the postmo-modelzoo, postmo-quantflow, vllm-postmo, and the adapted version of llama.cpp, vLLM.
For a postmo-modelzoo that is already open to the public, it's like a postmo-modelzoo."Real War Sample Room" for the open box.。
From quantitative compilation, performance evaluation, to final deployment, every step of the way, there's already code and tools that can be controlled directly.Examples of reasoning can be found across the three major platforms Linux, Windows 11 and Android.
It's just the model directory.15 major classesIt includes speech recognition, speech synthesis, large language models, visual language models, image generation, and full formatting.
In his own words, this open source is to do it."Best Practices Shared Without Reservation"。
More than 150 products bet on the same chip
More than 150 products are betting on the same M50, which means that the chip must be rolled at the same time.Agent Computer, smart mobile phones, robots and edge reasoningThere are four completely different connections.
Some devices are extremely energy-intensive, some wires don't plug, and others have to run on a purebred system.
Agent Computer: a 300-gram box running 12 billion parameters
Agent Computer, to put it simply,A computer specifically designed to run an AI AgentThe most representative of these is:Linkedin's AI host P7。
The host.With only 300 grams, one hand can hold it. The total power consumption is 30W, and a charger plug can take it away.
Even if the wires are removed, a large model with 122 billion parameters can run locally! 50 tokens per second are produced, the longest supporting 128K in the context.
It is equipped with the M50 chip.Typical power consumption is 10W, providing 160 TOPS of Int8 computing power and supporting up to 48GB of LPDDR5/5X memory.
The agent may need to run 7×24 hours and will continue to process emails and private files.
If you run it in the cloud, you pay a large token fee every month, and you don't feel comfortable handing out the data.
Instead of the P7,One-time hardware purchases and local reasoning is no longer required to pay for Token to the cloud.It's called the after tube."Token Factory by the Edge".。
It was created by Deepshard, a startup.Truffle 1It's just a matter of putting yourself in position."Privacy" on the Internet 24 hours a dayAIThe outer brain".This version, in collaboration with Motorola, also comes with the M50 chip.
You give it the details of your projects, the work habits you talk about every day, and you give it the details of your life.It's all on the device.After dealing with the same project, the agent can.It's a great way to connect these long-term memories.It's a great way to save yourself the trouble of switching backgrounds every time.
As soon as you click the authorization, it can dig itself into Slack and Gmail to help you search for messages, browse emails,Wait until you actually send a message or execute an operation, and then hand over your confirmation plate.
Even if you turn off the dialog box to do something else, back-end applications written by developers can keep up-to-date with the latest project progress.
For the development team, the tools are usually handy.If you go straight to the MCP, you can get in.It's not necessary to rewrite existing software, and it's not necessary to push it back.By adjusting the OpenAI-compatible interface, you can run it directly on the local model of the device.
This is also the scene where the M50 is plugged in by the Greens.FamilyAIHomeAgent HA100 ProIt's a lot of fun, and it's a lot of fun.AI is able to locally identify and organize without the need for cloud computing.
It's more fun to talk, more intelligent to play, and more fun to play.ClawHouse X1 gives Agent a customizable holographic imageIf you put it on your desk, it's a digital employee who helps you manage your schedule; if you move into the library, it's a learning partner who helps you practice conversation with your children.
Smart mobile terminals: a purebred copy of 35B's net run completely disrupted
In some units, computers are born without connecting to the internet.
In government, in financial securities, in these industries,"Offline availability" is a tough requirement.
The N90 Pro notebookIt's a full-fledged domestic route, equipped with the Galaxy V11 system.The 35B model runs on the M50 in a completely disconnected state, with 30 tokens per second.I'm not sure if I'm going to be able to do it online.
The M50 is capable of running a torpedo system because itThe mainstream operating systems andCPUArchitecture- Morrow says the M50 isIt's the only high-powered edge in the country to do this.AIThe chip.
Robots: AI managers who remember the habits of a family
When we get to the living room, another question is faced by the astrologers accompanying the robot.
It has to understand people's emotions and remember the family's life habits in their daily companionship.One of the moves in the living room is private data, and no one cares about getting it to the cloud.
The M50 is equipped with a 160-TOPS calculator.The star spirit keeps all emotional perception and long-term memory inside the body, and the data never leaves the house.
Edge Reasoning: A machine with 100 billion model lessons
What about a high school that wants to teach students to play big models and can't afford to put all the data and courses in the public cloud?
Education on the InternetIt's on a training machine, and it's inserted inside.After the LM5040 accelerator, the single-car integrates four M50s.
The entire machine can deliver more than 3200 TFLOPS and more than 1.5TB of memory!100 billion models can be deployed in schools locallyComputational management platforms and supporting courses are delivered.
From a 300-gram handheld box, a purebred state-owned notebook, to an accompanying robot in the living room and a campus server, the core is the same M50, but the models, systems, and scenes are very different.
Adaptation schemes and optimization parameters from the more than 150 projects have been stored in this open source software box by a later notebook.A real "bank of engineering experience" has been created.
And now the bank has officially opened its doors to all developers.
The warehouse is big enough and the road is wide enough.
The software can intersect so quickly that it can't break the cracks in the underlying structure of the chip.
In traditional chips, the data is stored in "warehouse" ("storage unit"), computed in "workshop" ("computing unit"), and the data is kept back and forth between the two sides.AIIt is estimated that 60% to 90% of electricity is consumed on this route.
In other words, wherever the data is, that's where it is.
However, it is not always possible to use this method.The chip area has been cut by about 70 percent, and the power consumption has been cut to about a tenth!
But that's only half the problem. Today's big model is divided into two main genres, the opposite of the "appetite" for hardware.
MoE (mixed expert) modelEvery time there are only a few "experts" who do the job, and the experts who don't do the job have to take their place, so what it takes is one.It's big enough".Warehouse"It's a very large capacity.。
Density and density modelsWell, on the contrary, almost all the parameters are nested every time you reason, and the data floods into the computing unit like a flood, so it needs to be one.The highway is wide enough, the bandwidth is very high.。
It's hard to come up with an architecture to do both at the same time.
This is the first time that a new technology has been introduced in the world.
2D CIM is about "big capacity, high computing power".It's also used in traditional 2D packaging, external memory such as GDDR, LPDDR5, and more.Specifically designed for MoE models and image and video generation.
3D CIM is dominated by "extreme bandwidth, high computing power".，Specifically designed for the Dense model.The bandwidth is following, and the calculators won't do the data.
When it comes to 3D, a lot of people are now doing 3D stacking, which is laying a layer of memory chips on a computing chip.
This is equivalent to moving the warehouse to the factory next door, and the road is short enough for data to enter the factory, or have to go back down the old water line.
And the 3D CIM that followed, even in factories.The flow of waterIt's all for you.
In its stacks,At the bottom of the logical chip, which is responsible for computing, is the SRAM storage unit.
Superimposed storage responsibilitiesNear-distance computing (including near-presence computing)This is the next chip.Local digestion (in storage computing)The two layers together reduce data transfer, thus releasing data.The doubling effect of "one plus one is greater than two".
It's not over yet.
Currently, 3D CIMs use DRAM, which can be replaced by cheaper, more energy-efficient Flash, RRAM.
At this year's global AI chip summit, Mor has revealed:The next generation of 3D CIM chips has achieved phased results.
The hardware is divided into two paths, but run by the same software.The developer learns a set of tools, replaces the chips, and makes a bit of adjustment to keep running.
The people who were doing OpenCL at the time, this time they've turned their homes on the ground.
Speaking of "open", the founder and CEO of MoSmart is an old acquaintance.
He worked at AMD headquarters earlier this year.The first generation of GPGPUDisplay cardsIt can also be used for general computing core research and development.It's the same.Core member of the OpenCL founding teamOpenCL itself is a set of open computing standards that anyone can use.
Later, he went to Meta to optimize the energy efficiency of the chip cluster and returned home as the CTO of the horizon.
Today, in the chip company he founded, he has also opened up the core of the software."Leadership in Open and Efficient Empowerment"。
From the core, to the end.
For the end-users, it's not the end of the world.Getting the source code is like holding the initiative in your own hands.
Do you want to add a single-door feature? Second development on existing tools.
Out of the Bug,The team can look at the code and the positioning issues themselves.You don't just have to wait for the manufacturer to fix the closed-source SDK (which only gives tools and doesn't give source code).
The company also said that it will respond quickly to bug feedback, new features, new requirements can be mentioned at any time.
Adaptation experience gained by new partners, optimized code, can also be added to the toolchain for later teams to replicate.
This time around, it's a group of partners - who make terminals, build systems, write frameworks, come up with a plan, connect, China's Great Wall, green lines, chat software, chat smart, etc. - who are back on stage and share their landing experiences.
The chips rise to the bottom of local computing, the open source software paves the way in the middle, the open ecosystem transforms the real-world experience of hundreds of projects into a public base.
It's a puzzle of three pieces.Make a side of itAIThe product threshold has been lowered to the ground.
From the core to the end.
When a chip company uses a chain of tools and real-world experiences outside of a chip as an open-source, edge-to-edge toolkit, it's the only way to get the most out of it.AIHe walked into the streets of thousands of homes and followed them as widely as possible.
RoboInd editorial channelResearch, robotics companies and market intelligence for cross-border sourcing.Explore the channel ↗
Share this
End of editorialExplore more from Global Robotics News ↗