One RTX 4090, with no hardware changed, and the specific Tensor Core computing throughput went from 165.6 TFLOPS to 331.6 TFLOPS, directly doubling!
What's even more surprising is that the potential of this excavation far exceeds the RTX 4090. RTX 3090 Ti, RTX 4070 Ti SUPER, and even the RTX 2080 Ti released in 2018, all achieved significant performance improvements in community testing.
All of this stems from the exploration by the first group of developers on GitHub of the underlying computing limitations of the Invidia GPU.
They found that the Tensor Core of some GeForce graphics cards had an instructional shutter speed limit when performing specific mixed-precision operations. By modifying the relevant configuration of the GPU start-up phase, the operation paths that were previously restricted could be restored to higher throughput.
There is no need to replace the GPU, and there is no need to press the graphics card BIOS. In verified Linux programs, the original NVIDIA drive can even run as normal.
Currently, projects for RTX 20, 30, and 40 third-generation graphics cards are available on GitHub, and some developers have completed experiments in the Windows 11 environment. An excavation around the "hidden computing power" of the Invidia graphics card is underway in the open source community.
RTX 4090 is doubled in power
The most notable results came from RTX 4090. In the open source project gpu-unlock-ada, developers JustAResearcher showcased the results of pre-unlock and post-unlock tests of the RTX 4090. In just 2 days, the project has accumulated 2.7k Star volumes.
Address of the project:https://github.com/JustAResearcher/gpu-unlock-ada
Specifically, in the Tensor Core operations of FP16 inputs, FP32 additions, the throughput rose from 165.6 TFLOPS to 331.6 TFLOPS, almost doubling. The same change occurred in the BF16 inputs, FP32 additions and tests, rising from 165.5 TFLOPS to 330.6 TFLOPS.
In order to rule out coincidences, the developers also increased the number of test cycles from 20,000 to 40,000 times, resulting in the FP16/FP32 and BF16/FP32 throughput rates of 330.8 TFLOPS and 330.3 TFLOPS, respectively, essentially consistent with the previous results.
It is also worth noting that the entire experiment did not replace the graphics card hardware, nor did it achieve performance improvements by increasing the power limit, manually speeding up or modifying the official NVIDIA drive.
According to technical reports, the test graphics card uses an AD102 chip with 128 SM-based processors running the Linux 7.2.4 kernel with a maximum power output of 450W. The developers used the GPU startup-stage solid-state execution process to open the write permissions of protected registers and then modify the register configuration to control computing speeds.
Upon completion of the configuration, the graphics card underwent a PCIe functionality-level reset (FLR) followed by normal takeover by unmodified NVIDIA 615.78.08 drives. After drive initialization and CUDA testing, registry checks showed that the unlock configuration was still in effect. The developers successfully removed the RTX 4090 speed limit in some Tensor Core operating modes, allowing the original computing unit to achieve a higher throughput rate when executing FP16/BF16 inputs, and FP32 cumulative commands.
However, performance changes did not cover all operating types. For example, FP16 inputs had 325.7 TFLOPS in aggregate throughput before unlocking and 331.6 TFLOPS after a longer loop test after unlocking, and no similar doubling occurred. The IT8 operating throughput was basically maintained at around 661 TOPS, while the normal FP32 parametric operation remained at about 79.1 TFLOPS. This indicates that the lock mainly affected specific Tensor Core operating paths that were previously speed-restricted and did not double all the computing capabilities of the RTX 4090.
At the same time, another RTX 4090 holder completed the independent replication using the reviewed project code. The experiment used Linux 7.2.11-arch1-1 kernel and NVIDIA 615.78.08 drives, using the same graphics card before unlocking, the same test program and 450W power limit, further verifying the results. The test showed that the FP16 input, FP32 plus command throughput increased from 160.4 TFLOPS to 320.4 TFLOPS, BF16 input, and FP32 plus output increased from 159.7 TFLOPS to 320.2 TFLOPS, and both results nearly doubled.
The following is the address of the test:
In matrix multiplication tests using cuBLAS, the throughput of the FP16 operations improved by about 12% to 34% depending on the matrix size. Of these, the throughput of the 4096×4096 matrix multiplication increased from 153.2 TFLOPS to 206.0 TFLOPS, an increase of about 34.4%. The performance of the BF16 varied considerably. The 1024×1024 and 8192×8192 tests improved by about 19.6% and 16.0%, respectively, but in the 4096×4096 tests, the performance remained essentially unchanged.
This difference relates to the way actual computing tasks are performed. Testing the underlying instruction throughput reduces the impact of memory access as much as possible, while real-matrix multiplication also involves data transfer, computational scheduling, and operational efficiency at different scales. Thus, the RTX 4090 specific Tensor Core instruction throughput doubling is supported independently, but this does not mean that actual AI training or reasoning speeds can also be doubled.
The RTX 30 series is equally powerful.
The RTX 30 series has also received good news. Developers Zagitta's open-source project gpu-unlock has been successful on multiple RTX 30 series graphics cards.
Among them, RTX 3090 Ti performed quite well. At the 450W power limit, the BF16 matrix operating throughput increased from 87.5 TFLOPS to 157.4 TFLOPS, an increase of almost 80%; the TF32 improved to 94%, almost doubling.
RTX 3080, 3070 and 3060 also received reports of successful unlocking. Among them, the RTX 3080 10GB BF16 throughput increased from 65.4 TFLOPS to 113.4 TFLOPS, an increase of about 73%; related tests for RTX 3070 and RTX 3060 also achieved improvements of about 81% and 76% respectively.
Address of the project:https://github.com/Zagitta/gpu-unlock
Similar to the previous RTX 4090 unlock, gpu-unlock also uses the GPU startup-stage firmware execution process to modify the register configuration that controls the Tensor Core instruction release rate.
Specifically, NVIDIA GPUs contain registers for controlling the speed at which different instructions are executed, including SS0 and SS1. These registers are protected by permissions and cannot be directly modified by ordinary drives.
Zagitta took advantage of the V67 technology in the previous CMP graphics card unlocking project to take advantage of the vulnerabilities in the GPU Safe Boot Fixture to obtain authorization-level masks for the corresponding registers by re-routing the ROP and modifying the PLM. The program then set SS0 to 0*888888 and SS1 to 0*8, allowing the corresponding operation path to be switched to a higher instruction release rate.
However, Zagitta did not adopt a modified NVIDIA drive method, but integrated the entire unlocking process into the Linux kernel. When the system is started, the module runs before the official NVIDIA drive receiver graphics card, completes the registry configuration modification, and then resets the configuration via the PCIe function level (FLR). Unmodified NVIDIA drives can then be loaded normally.
As the technology spread, more and more graphics card users began submitting independent test results.
A developer using RTX 3080 10GB was further tested at the 320W power limit. In addition to an increase in BF16 throughput of about 73%, the matrix operation throughput of FP16 inputs plus FP32 was increased from 64.88 TFLOPS to 110.14 TFLOPS, an increase of about 70%.
In the 120-second FP16/BF16 alternating matrix computation test, the number of times the chip was completed increased from 6940 to 11860 times, increasing the average throughput by about 71%. Both tests maintained power at about 318W, with peak temperatures of 78°C and 79°C respectively.
The test also verified the accuracy of the calculation results. In FP16, BF16 and TF32 matrix operations, the output before and after unlocking is completely consistent. However, not all computing tasks will experience a similar improvement.
In addition to matrix operations testing, some actual AI tasks also gained acceleration. One RTX 3090 user who used vLLM to run the 27B BF16 large model found that the prompt word prefill speed increased by about 33%, and the response time for the first Token shortened by about 24%. Another developer who used two expandable versions of RTX 3080 also achieved performance improvements of about 33% to 36% in the large model prefill test.
RTX 3080 test report:
However, there was no significant change in the speed of Token generation in the two sets of experiments. This is due to the large-scale matrix computation required during the pre-filing phase of the model, while Token generation is often more limited by the display storage bandwidth.
From the modification of underlying registers to real AI workloads, the unlocking has resulted in significant performance gains on multiple RTX 30 series graphics cards. The same technical thinking has begun to be applied by developers to earlier GPU architectures.
The potential of 2080 Ti was also discovered eight years ago.
The development of the RTX 20 series is even more interesting. The RTX 2080 Ti, which was released in 2018, is now eight years old. The flagship chip sold for thousands of dollars at the time, has long since fallen out of the top tier of high-end GPU performance competition. However, with the spread of NVIDIA GPU algorithmic unlocking technology, the old chip was again tested by developers.
The developer nlzy ported the relevant technology for gpu-unlock to the NVIDIA Turing architecture and opened the project turing-unlock on GitHub.
Address of the project:https://github.com/nlzy/turing-unlock
The developer tested a Dell RTX 2080 Ti with a non-standard storage configuration of 22GB. The original RTX 2080 Ti had only 11GB of storage, so the hardware configuration of this test card differed from the normal retail version. Under the 250W power consumption setting, the developer used cuBLAS to perform matrix multiplication tests with FP16 input, FP32 added. When the three-dimensional M, NK matrix was up to 2048, the throughput increased from 55.7 TFLOPS before unlocking to 91.4 TFLOPS, an increase of about 64.1%.[3]
Other matrix scales have also been upgraded to varying degrees:
1024: Increased from 50.3 to 68.3 TFLOPS, an increase of about 35.8%;
4096: From 53.7 to 81.7 TFLOPS, an increase of about 52.1%;
8192: From 52.8 to 77.9 TFLOPS, an increase of about 47.5%.
The RTX 2080 Ti corresponding to the FP16 Tensor Core peak throughput, given by NVIDIA in the Turing architecture white paper, is 56.9 TFLOPS. According to this calculation, the test results before unlocking on the 2048 scale reached a nominal peak of 97.9%, and after unlocking reached 160.6%. This indicates that the original speed configuration did limit the throughput that this computing path could play.
Similar results were seen in another developer's experiment.
GitHub user haohervchb has released a separate Turing-unlock research project to test an ASUS RTX 2080 Ti. The graphics card uses a TU102 chip with 68 SMs and a standard 11GB of display capacity. Unlike nlzy's Linux kernel module, the project uses UEFI execution before the system starts and modifies the GPU's configuration before the operating system is loaded.
In the FP16 input MMA (matrix multiplication) instruction test for FP32, the pre-unlock throughput was 46.94 TFLOPS, which nearly doubled to 93.66 TFLOPS after unlocking. Measured by the number of float operations performed per SM, per clock cycle, the performance increased from about 511.7 FLOP/SM/clock to 1022.4 FLOP/SM/clock.
Address of the project:https://github.com/haohervchb/Turing-unlock
In the 8192×8192 scale GEMM test, the throughput increased from 48.15 TFLOPS to 85.17 TFLOPS, an increase of about 76.9%. Although it did not reach nearly double the level in the bottom-order test, this is still a considerable improvement for a graphics card released eight years ago.
Currently, the scope of validation for both studies remains limited. The evidence available only suggests that some of the TU102 graphics cards can improve the throughput of specific Tensor Core operations by modifying the speed configuration. Whether other RTX 20 series models can use the same method, as well as compatibility under different driven and fixed versions, needs further verification.
Windows 11 has also been successful.
The above exploration is mainly focused on the Linux environment. The Windows platform has also made experimental progress with the successive open source development of several projects.
Developers TOTORONG released the AD103_win_unlocker on Hugging Face, showing experiments to modify the GPU computing configuration in the Windows 11 environment.
Address of the project:https://huggingface.co/TOTORONG/AD103_win_unlocker/blob/main/docs/INSTALL.en.md
The test uses an RTX 4080 with an AD103 GPU with an expandable storage capacity of up to 32GB.
Unlike previous Linux kernel modules, it uses the UEFI program to execute configuration modifications before starting Windows and then transfer control back to Windows.
The project's published test results show that in a matrix multiplication of the 4096 scale:
FP16/FP32：91.54 → 182.80 TFLOPS。
BF16/FP32：91.91 → 184.49 TFLOPS。
Both tests have nearly doubled their throughput. This means that the underlying configuration changes are not limited to Linux systems.
However, the current results of Windows 11 still come from specific modified graphics cards and cannot be used as a basis for general compatibility with the regular RTX 4080 or the entire RTX 40 series.
The link is as follows:
© THE