Posted on Leave a comment

Review the review

Why a single incredible review is putting a serious risk to our existence as developers and why isn’t doing any favour to the X-Plane community and to its author credibility.

With great surprise I’ve read last night what Sérgio Costa patron of wrote about the AW109 I gave him for free in a review that can be easily defined as catastrophic. No one better than HeliSimmer has represented in recent years a pivot for the X-Plane helicopter community, but this time the review has gone far beyond imagination, destroying with rage almost every single piece of a brand new product as the worst ever.

Payware or not, every product is subject to judgment, but this has been misrepresented by opinions rather than facts far from the objective result, opinions that simply don’t tell the truth to the readers and end up having an impact not only on our sales but also on the credibility of their author.

I know what you’re thinking, this may be sort of an emotional protest for a bad rating, but the product sincerely deserved better, the community deserve other than reading about real pilots behind a home computer, three years of work and dedication deserve better than a poor 4/10 score. Let’s see why, let’s try to review the review one opinion at a time.

Our AW109 version 1.00 in all of her beauty – video by Catchy Mussel

X-Plane 12 only features
Wait! No one here has ever stated that the product is only compatible with X-Plane 12 and by saying that you are misleading your readers. When we write about CAS visiblity we talk about the same bug (XPD-10211) that afflicts the 747-400 and AW139, known since 2018 as we were the first to report and that only recently – Ben Supnik wrote to us in november – has been estimated in terms of resolution time. We wanted to make a transparency operation and tell the truth, nothing more. And no, it’s not like an application for Windows 11, it’s an X-Plane BUG – yes, it has bugs too, just like any other software – not our fault and definetely something the users needed to be aware of before purchasing the product.

Model and textures
Here is the most useless part of the review. What Sérgio seems not knowing is that there is a limit to the number of polygons a developer can think of putting into a 3D product for home PCs. The greater the number of vertices, the lower the performance on the GPU. Having said that, our new Blender-based pipeline, other than providing an upstream measurement based on individual objects and cumulative weight, has allowed an exponential increase of the meshes appearance compared to the previous cycle (the AW139, to say). When he blather about polygon numbers and say that the cyclic tube (!) has areas for improvement, he seems willing not to do a constructive review but the exact opposite, making a personal impression to became a defect! Speaking about polygons, should they mean something to the reader, the AW109 fuseleage has over 644.000/145 vertices/objects, against the 170.000/317 of the AW139.

Saying that the animations are weird (!) doesn’t tell the truth either, unless the only animation good to go is the poor one on the default S76 rotor, which is fluid because it has zero impact on GPU but in return it’s horrible. The AW109 swashplate, each single blade movement around it, the overall effect are something we dedicated months and the result is appropriate and in line with other similar products. Maybe suitable for critics, but still proper for a first release. Instead he goes analyzing infinitesimal details like talking about a high-end simulator worth millions of Euros, stumbling over an incredible lack of objectivity, like he wanted to tear the product apart. Austin Meyer himself – the X-Plane creator – tried the AW109 saying it’s really fine and good looking. For sure there are areas for improvement, like everything, but they are part of the progressive process of any software or game or whatever program for the office. Instead, cherry-picking is done on a endless myriad of things that lined up cannot do nothing but discourage the purchase and that’s exactly what’s happening, fatally discouraging us from keeping it up!

Here I must say I expected it, we weren’t 100% satisfied so the evaluation is ok, but we knew that. However, at the time of this article we already started to improve, particulary with the 1.02 release.

Pretty much all the rest
Asking real pilots – in this case as many as two – to test a videogame – that’s what it is to many – poses several problems. First of all because the presence of a real pilot represents the (incorrect) assumption that a correspondence between the experience of the real pilot and that of a gamer must be found, which obviously is not possible on a home PC and only really relevant in professional, hugely diverse and hugely expensive contexts. The experience with the home simulator, on the other hand, is extremely subjective and if as a real pilot you are saying that the product is crap, you are telling a lie, not only because it’s not but because you’re in a different context.

The second problem is the expectation of the reader, something in the heart of the sim business that we and Sergio knows well. The average user thinks “Wow! A real pilot, that must mean these guys are serious!” and ends up forgetting the unbridgeable distance made up of risk, study and other little problems of liability which are not in question here. But in the meantime he has read the article, spent his money, made impressions and bring conversions – read real money – to the website where the article was published.

And here we introduce a third problem, which is that – once more – of the credibility of those who put real pilots behind the reviews, as they (rightfully) don’t stop to evaluate the result for what it is, on a simmer POV, but instead they wheigh some homemade software against a real thing: a failure without appeal, of course.

After having analyzed the AW109 under the microscope, before pleading the accused guilty and sentencing him to death by openly discouraging everyone from buying it, he produces an incredible flashback on the AW139 and some buzzword that has nothing to do with the product, with the sole purpose of supporting the unsustainable thesis that not even the AW109 is finished! We remain speechless, certain that, net of all considerations, the community deserves more than an indictment, built on the pretext of doing everyone a favor, but that doesn’t do anyone any favors.

Making clamour by destroying the work of a tiny developer might even make you pay some bills, but it will definitely make the community poorer. And this ridiculous review proves that we’ve been maybe looking at the wrong resource for the X-Plane helicopter scene.

The Rating
As said in the opening, our AW109 scored an incredible 4/10 (!) Also incredible is that the arithmetic average of each of Sergio’s ratings (39/6) scores a 6,5 (which isn’t that bad) not a miserable 4. Here’s a screenshot from his website.

There’s no relation between the total score and each single rating

The rating logic isn’t something secondary in a website that does business by reviewing commercial products, not because it may ruin someone’s life – as it’s happening – but essentially because an unprecise, arbirary, clearly wrong rating won’t make that website trustful. Asking Sérgio about this, he replied that the score is not the average, but each rating has a different weight and that’s the problem, as the rating seems nothing but arbitrary.

To counter-prove this, we’ve been sending a couple of surveys to our current customers, and the suprising results will be published here.

In conclusion, the review was unfair as it was based on assumptions, lack of objectivity, opaque logic. Beside that, a review should serve to improve a product, certainly not to take a developer off the market. For each of this reasons this has been the very last time he got one of our product to review and I strongly discourage any developer letting him to do it. Definetely not as credible as he might want us to believe.