It’s barely been a day since OnePlus launched its latest flagship OnePlus 5, and the company is already in trouble. A report from XDA Developers claims that OnePlus employed a method to target popular benchmarking apps on Android, allowing the device to post better scores. Earlier this year, the same XDA Developers team came up with a report stating that OnePlus manipulated OxygenOS builds to gain a higher score with the same benchmarks, and it looks like the manufacturer hasn’t learned from its mistakes with the OnePlus 5.
Last month, we reported a screenshot of the OnePlus 5’s Geekbench benchmark score that was making the rounds online. That led to us wondering if the OnePlus 5 can beat the Samsung Galaxy S8 when it comes to benchmarks. After yesterday’s launch, a handful of reputed websites published their OnePlus 5 review, with the benchmark scores showing that it indeed surges past the S8 in Geekbench.
The XDA Developers‘ report openly states that the benchmark scores on the early OnePlus 5 reviews cannot be trusted, as the review units came with software that allowed the devices to cheat at apps like Geekbench. Let’s take a quick look at how XDA figured out OnePlus 5 was manipulating benchmarks.
How does it work?
Synthetic benchmark scores don’t really matter anymore, but they’re great for marketing, and as such it is a big deal to get a high score. The idea is to include an auto-triggering mechanism to juice out the maximum performance from the Qualcomm Snapdragon 835 chipset to achieve the highest possible scores in the benchmarks.
The 10nm FinFET Qualcomm Snapdragon 835 has eight Kryo 280 processing cores. These eight cores are bifurcated into four high-performance cores clocking up to 2.45GHz (big cluster) and four high-efficiency cores (little) clocking up to 1.9GHz. This arrangement is similar to ARM’s big.LITTLE architecture design — putting four performance cores in a big cluster and four efficiency cores in a little cluster.
Launching a benchmark app triggers the alleged mechanism to kick in on the OnePlus 5, and the frequency of the four high-efficiency cores jumps to 1.9GHz. With that sustained clock speed, the OnePlus 5 is shown to gain highest scores in GeekBench 4 tests. In fact, those scores were higher than those achieved by other phones carrying the same Snapdragon 835 chipset.
So the XDA Developers team ran the GeekBench 4 twice but with two separate builds – One was downloaded from the Google Play Store and the other was their own build. That resulted in two diverse set of multi-core scores. The team further explained the difference graphically where the hidden build benchmark had only 24.4 percent of readings returned to a maximum frequency of 1.9GHz for the little cluster. In simpler terms, on polling the CPU frequency at 100 milliseconds rate, the little cluster hit the maximum frequency state only about 25 times during the benchmark test.
Meanwhile, the GeekBench 4 build from the Play Store hit the maximum frequency state during 95 percent of the reading, as the little cluster’s processing cores ran at 1.9GHz.
So OnePlus is alleged of keeping the CPU frequencies of its high-efficiency cluster to the maximum to achieve a higher multi-core test score. However, the same wasn’t the case with single core tests, which showed only marginal difference through the benchmarks.
Lists of benchmarking apps affected:
The behaviour of OxygenOS is tuned only to trigger only when one summons a particular benchmark on the OnePlus 5. Here is a list of the benchmark apps which are reportedly unable to detect the trickery:
- AnTuTu (com.antutu.benchmark.full)
- Androbench (com.andromeda.androbench2)
- Geekbench 4 (com.primatelabs.geekbench)
- GFXBench (com.glbenchmark.glbenchmark27)
- Quadrant (com.aurorasoftworks.quadrant.ui.standard)
- Nenamark 2 (se.nena.nenamark2)
- Vellamo (com.quicinc.vellamo)
Why it matters:
Benchmarking apps exist to give a fair estimate of the potential that a chipset, or a device can offer. It’s only in recent years that the benchmark scores gained prominence at phone launch events, and only then just a small subset of users care about them. At the same time, methods to gauge and measure the real-world performance, as well as power consumption, continues to be a much better indication of how a phone’s prowess.
The XDA Developers team reached out to OnePlus for a comment, and received this:
People use benchmark apps in order to ascertain the performance of their device, and we want users to see the true performance of the OnePlus 5. Therefore, we have allowed benchmark apps to run in a state similar to daily usage, including the running of resource intensive apps and games. Additionally, when launching apps the OnePlus 5 runs at a similar state in order to increase the speed in which apps open. We are not overclocking the device, rather we are displaying the performance potential of the OnePlus 5.
Do you look at the benchmark scores of a chipset and a phone before you decide to purchase it? Will such tactics make you change the way you look at benchmarks? Share your thoughts in the comments section below.
The post OnePlus once again accused of benchmark cheating — this time on the OnePlus 5 appeared first on Android News.