2007年の『龍が如く 見参！』以降『龍が如く３』『龍が如く4 伝説を継ぐもの』『龍が如く OF THE END』まで社外のスタジオで役者さんを撮影し、メッシュデータもそのスタジオで生成されたものを使用していました。 3Dスキャンを行うようになりキャラクターのモデリング時間は40％削減でき、クオリティも上がりました。
最初からシンプルになるように組んだことが功を奏しています。なぜなら、ボトルネックのトップとしてUE4の標準関数(Set Actor Location And Rotation)が挙がっており、自作の関数たちは次点以下になっているからです。自作の関数BP_BallisticObject.Updateが次点ですので、ここはもちろん考察しておくべきですね。
上記に書いた通り、Blueprintで組んだカリング処理が重くなっては意味がありません。ある意味、C++で組まれたUE4標準関数Set Actor Location And Rotationに勝てるような、できる限りシンプルなカリング処理を選ぶ必要があります。そこで今回は「内積１発でざっくり判定」する方法を選びました。
クラスメンバ変数Location (OptTest)とRotation (OptTest)を用意。修正前ではGet Actor Locationから取得し、Set Actor Location And Rotation でセットしていたところを、いったんこの変数に保存している。これは、Set Actor Location And Rotation を呼ばなくても、Tickのたびに更新される位置と回転を保持しておくため。
修正前ではSet Actor Location And Rotationを呼び出していた場所で、カスタムイベント「TickEvent Cull If Outsight」を呼び出している。このカスタムイベントについては後述する。
その他の個所については、変更なし。
カスタムイベント TickEvent Cull If Outsight
TickEvent Cull If Outsight： プレイヤー機と敵の弾のUpベクトル同士の内積の大小を判定しているところ。Math Expressionの 0.7071は cos(45度)の値。つまりプレイヤー機のUpベクトルとの角度差が45度以内のものを可視と判定している。
SubEvent Cull If Outsight ＞ Sequence - Then 0： 可視圏内／可視圏外へ切り替わった時の処理。SweepのOFFと、コリジョンのON/OFF、アクタの表示／非表示を切り替えている。可視圏内に入った直後はSweepをOFFにしておかないと、過去の可視圏内だった位置から可視圏内に戻った現在位置まで、長いSweepが作られてしまい、意図せぬコリジョンが発生してしまう。
SubEvent Cull If Outsight ＞ Sequence - Then 1： 可視であれば Set Actor Location And Rotation を呼び出した後、SweepフラグをTrueにしている。
アクタの数やTickの数を必要最低限に絞る： この数は純粋にCPU負荷に影響します。今回の弾幕のようにアクタ数が多いと、明確に分かりやすく処理落ちし始めます。まずは、アクタの数を必要最低限に絞ることです。また、Set Actor Tick Enabledなどを使って、不要なTickは切ってしまいましょう。
Set Actor Transform系はTick１回につき１回まで： 前述したとおり、一見トランスフォームをセットするだけのように見えて、その中ではコリジョンのSweep処理や、コンポーネントの座標変換など多くの処理が行われています。位置や回転などのトランスフォーム値は変数で計算しておき、確定した段階でSet Actor Transform系を呼び出すようにしましょう。また、不要なSweepは行わないこと、コンポーネントの数を絞ることも大事です。
ここで、Blueprintでの最適化と、手動でのC++化が、同レベルで天秤に掛かっていることを不思議に思う方もいらっしゃるでしょう。一見、手動C++化で決まりだと思えます。しかし、私が手動によるC++化を避けている理由は、関連するゲームプログラム周りの設計を再構成する必要がでてしまいやすい点にあります。例えば、ゲームバランスに直結するようなBlueprintを安易にC++化してしまうと、以後ゲームバランスを変更するたびに Visual Studio でのビルドが必要になり、作業スピードが落ちてしまいます。また、アーティストやレベルデザイナーが頻繁に触っているBlueprintをC++化するわけにもいきません。そのためには、基底クラスをC++で作って、そのBlueprintの親にする等、何を表(Blueprint)に出して、何を裏(C++)に隠すかを設計しなおさなくてはならないのです。一方のNativizeはパッケージングにのみ影響する機能ですから、その辺の心配はいりません。
Hello. I'm Naoki Sakaue. I work as a QA engineer in "Ryu Ga Gotoku Studio" of SEGA Games. I have been creating the video games of "Ryu ga Gotoku/Yakuza" series about 10 years. In the following article, I will explain the role of a QA engineer and playback the game development history of "Ryu Ga Gotoku Studio".
Before we go on to the main topic, let me explain what a QA in the game development is. QA stands for Quality Assurance. When we use a word "QA" in the game development, it usually means the work itself but also the departments or the project teams, depending on the organization. The basic purpose of QA in the game development is to find bugs and make sure if the games are working correctly in required quality.
Then, what do we do when we use a word "Test" in the game development? In most cases, it means to play the games manually to find bugs and fix them. (Unit testing and smoke testing which are well-known in the software development are also held in the game development, and I will explain them later in this article.)
The games under development are checked by people called a "QA Tester" to see if there is no bugs left in the games, the products are well followed by the specification documents, the games meet the product standard, and the games are come up fun as the final products. Their works cover a wide range, such as a simple testing of crashing a wall to find collision bugs, finding failures triggered by a complex procedure or specific timing, and a physical testing of plugging or unplugging cables.
It is likely that people think of game testing as just a simple process of finding bugs, but it is only a step to QA and not a goal. The main purpose of game QA is to assure that the games are working correctly in required quality.
So, this process is very important that the games cannot be made into the final products without it!
Works of a QA Engineer
Recently, as the game development is getting bigger and bigger in size, it’s becoming more and more difficult to test and develop the games manually. To solve such problems by engineering technology, a new profession called a "QA Engineer" (like me) was born. What is required for a QA engineer might differ by organization. At our office "Ryu Ga Gotoku Studio", the QA engineers make efforts to automate the environment of the game development and QA and to increase its productivity every day.
At our studio, to improve the productivity of the game developing environment, we automate build and convert of game assets, and improve efficiency for the workflow of issue tracker (Trac/Redmine/JIRA etc.). In some offices, people called a "Build Engineer" do these jobs. But at our studio, the QA engineers usually take these roles, and sometimes ask for help from other engineers of our team or different departments, depending on the situation.
In this article, I will mainly explain the game testing on my work and can give you only a brief explanation of the automation technology for the game developing environment. So, if you are interested in the automation technology and want to know further information, please check out the following link of Japan Symposium on Software Testing in Tokyo 2016 (JaSST'16 Tokyo).
From here, I will move on to the next topic "Automated Testing", which is one of the main works of a QA engineer. I will explain how we came up with a new profession “QA Engineer” at “Ryu Ga Gotoku Studio” through a history of “Ryu ga Gotoku/Yakuza” series.
We have been releasing "Ryu ga Gotoku/Yakuza" series every single year in Japan, and the latest title which was released this year is the 14th. To release the mass scale games in such a high speed, at “Ryu Ga Gotoku Studio”, each one of engineers and a team as a whole always have been thinking what we can do to automate the game developing environment and to enhance its productivity. This spirit gave a birth to automated playtesting!
When we had started developing games for PS3 generation, we had developed our original automated playtesting system “AutoTest” and started operating it. This is a system which enabled PC and development machines to run through automatically throughout the midnight, so that they can detect errors even after the engineers left the office.
In the following figure (Figure 1), "AutoTest" gets the latest build and starts to play the game automatically, and Player AI (like bot) can emulate pushing buttons and play the game automatically.
Figure 1: The Architecture of "AutoTest" System
At the beginning, we tried to make the player in the game move randomly. We made them crash into a wall to find collision bugs. And then, we gradually expanded the area of automated testing, and now that we can clear the main story of the games with the replay data on the basis of playing the games in manual testing! Therefore "AutoTest" can expand the area of automated testing.
We kept brushing up the functions of "AutoTest" even further. In the game of “Fist of the North Star: Lost Paradise”, "AutoTest" cleared the main story all by itself. For “Fist of the North Star: Lost Paradise”, we made additional functions to automated testing which can cope with widely varying situation, and now that it enables us to find various types of bugs. For example, "AutoTest" can select the bot AI that mashes the decide-button during dialogue scenes, and changes to the battle’s bot AI when it attacks enemies (Figure 2). It can select bot AI that meets the circumstance of the game.
Figure 2: "AutoTest" Demo (Fist of the North Star: Lost Paradise)
Furthermore, an effective cooperation system is built up between manual testing and automated testing, so that "AutoTest" can take charge of testing numerous combination steps that can be hardly handled in manual tests.
You may think that the more the automated playtesting finds various patterns of bugs efficiently, the better works we accomplish as a QA engineer. But it is preferable that AI technology concentrates on removing only easily-find-bugs out, while we take our time to find more complex bugs, because we always want to focus on improving quality of our games and create the best and the most fun games ever before. The automated playtesting does support and help the manual testing, but not all the process of game testing can be easily replaced by the automated testing.
(2013-) The Birth of a QA Engineer and the Acceleration of Automated Testing
We have been improving and adding new functions to the automated environment for the game development and QA every time we release "Ryu ga Gotoku/Yakuza" series at our studio “Ryu Ga Gotoku Studio”. But on the one hand, it contributes to speed up the game development and improve the game quality, on the other hand, maintaining the automated environment increased the burden on developers. Therefore, I stepped forward and started to take in charge of the QA engineering since the development of "Ryu ga gotoku Ishin!" which was released in 2014 in Japan.
It was a wise decision to assign an exclusive QA engineer in a developing team that it carried out a great evolution to QA engineering technology at unbelievable speed on a daily basis. We reorganized the original developing environment and started integrating the systems in order to manage and maintain them easier. We also expanded the functions of “AutoTest”, and moreover we implemented the new functions for log analysis which I will explain in the next paragraph.QA engineering technology of “Ryu Ga Gotoku Studio” is still continuing evolving and it accelerates its speed even today.
(2015-) Analysis of Testing Result
At our studio, we assigned an exclusive engineer and we can now handle and manage the automated testing much easier. But there, we faced another new problem. Other than "AutoTest", we use Jenkins for automated testing. (Of course we use it as a continuous integration tool.) But it can only tell us either the result of the test is a success or a failure. We wanted the results in much more detail. so we started to analyze the logs of testing since "Ryu ga Gotoku KIWAMI" (Released in 2016 in Japan/The English version is "Yakuza Kiwami") and "Ryu ga Gotoku 6: Inochi no Uta" (Released in 2016 in Japan/The English version is "Yakuza 6: The Song of Life").
The following graphs (Figure 3 and 4) show the visualization of heap maps which are created by collecting usage of VRAM (Video RAM/Graphics Memory) (Figure 3) and collision errors in "AutoTest" (Figure 4).
Figure 3: Heat Map for Usage of VRAM (Fist of the North Star: Lost Paradise)
The heat map below visualizes the usage of VRAM in the city of "Eden". Red is high usage, and X is NG (over 100% usage).
Figure 4: Collision Error Map in Very Huge "Wasteland" (Fist of the North Star: Lost Paradise)
As a result of making the “AutoTest” play the games, defects other than easily-found-bugs, such as warnings, were found, just like when we play the games by ourselves. And also it can check the game performance for us. By collecting the logs and visualizing them, we can now save a lot of time and fix bugs much faster with high accuracy.
Moreover, we analyzed logs of manual playtests and could get feedback of game level design. It enabled us to create more fun games. The following graphs (Figure 5 and 6) are examples of feedback of game level design to analyze logs of manual tests (Figure 5) and "AutoTest" (Figure 6).
Figure 5: The Game Over Points of Manual Playtesting (Yakuza 6: The Song of Life)
We visualized points where player’s HP is zero in manual tests. We could make use of it for game level design.
Figure 6: Item Drop Odds of Automated Playtesting (Fist of the North Star: Lost Paradise)
We measured odds of items that the enemies dropped in automated playtests. We confirmed the item odds are correct and the enemies can drop the rare items for game level design.
If you want to know about our automated playtesting and playlog analysis in more detail, please take a look at the following link of Japan Symposium on Software Testing in Tokyo 2018 (JaSST'18 Tokyo)
(2018-) Thought of Test Pyramid and QA Engineering in the Future
No matter how efficiently we became to analyze the result of automated testing, we can never stop improving the QA environment. Nowadays, AI technology is a big trend everywhere around, and the efficiency of the automation technology and the log analysis in the game development which have been based on our experience are expected to make much more progress with an effect of AI technology.
The test pyramid is often used to show the execution of automation efficiency. In system software or web development, the test pyramid is desired to be like the figure below (Figure 7).
Figure 7: Test Pyramid of System and Web Development
UI Testing
The product testing. It is high cost and difficult to automate.
Integration Testing
Testing if the combination of individual functions and modules work correctly.
Unit Testing
Testing of individual functions and modules. It is the lowest-cost testing in this test pyramid.
By increasing the ratio of lower-cost unit testing and automating it, it is possible to cut cost and improve efficiency. Then what about the test pyramid in the game development? In most cases, it is likely to be a pyramid below (Figure 8).
Figure 8: Test Pyramid of Game Development (Real)
Manual Playtesting
Testing to be played by hand.
Smoke Testing
Simple testing to check if the games launch and the individual modules work correctly. The shorter the time, the better testing. (about a few minutes)
Engine and Library Testing
Testing if samples of the engine and the library work correctly when they are developed in the company.
Assets Testing
Individual testing if the game assets (the models, the textures and the motions etc.) work correctly.
As you can see from the figure above (Figure 8), the pyramid is upside down and very unbalanced. The ratio of the manual testing at the last stage of the game development is very high, which means it is inefficient and high costs. In the game development, these testing relies on human resources that it is difficult to increase personnels in a short period of time, especially under such a current circumstance of labor shortage.
And because of its data-driven architecture in the game development, it always seems to be very buggy by data(assets). Sometimes the number of these bugs is even greater than that of programming bugs.
So, we incorporated unit testing before final testing. You may think “Why do we need unit testing in the game development?”, but once you know that it can detect errors and check consistency of individual assets before the final tests, you may easily imagine how extra works and retake-costs we can eliminate by having unit testing.
We have been trying to make further efforts to fix the inverted triangle to a well-balanced triangle. For example, except for “AutoTest” and log analysis which I have already explained, we strengthen detecting errors in assets-converting and increasing smoke testing.
Figure 9: Test Pyramid of Game Development (Ideal)
With the ideal pyramid above (Figure 9), you may wonder why we had started to automate playtesting which is highest-cost testing first.
There are two reasons for it. Because the results of unit testing in the games can be observed only in the graphics output, it is difficult to determine if the result is a success or a failure. And the other reason is that the members of the development teams accept errors of automated testing as errors we must fix.
The second reason is especially important. In a testing process, once an error is found, the error must be fixed right away. But if the errors which automated testing found are not trusted by the developers, they would have a question such as “Do we really encounter the same errors in the product?”, and that these bugs would be possibly ignored and thus left over in the games. In this case, we will end up in just wasting time and labor. We should build the test environment exactly like the product which surely ensures the reliability of automated testing. It is important to raise the organization culture that every time the automated testing finds errors, we must fix them immediately.
At “Ryu Ga Gotoku Studio”, every team member recognizes the need of the “Automated Playtesting” and trusts the system, thus we are able to keep increasing the ratio of low-cost automated testing. If you are willing to try the automated testing, my advice for you is to build automated playtests which enable a player in the game move randomly by mashing buttons, and then gradually share them with your teammates.
Conclusion
As you can see from this article, QA engineering is very essential technology in the game development now, and it is desired and expected to evolve greater in the future.
Talking about my career, I first started working as a game programmer, and then experienced game development and QA as a developer. That’s why I can easily tell what is exactly desired for QA and what should be automated in the game development. In addition, I can suggest the concrete solutions for automation of the game development and seek for better productivity. In this sense, I believe that I’m making the most of my career. If you are interested in QA, a QA engineer would be a good choice for whom already have experienced game creating as a programmer.
We have been trying to make a comfortable environment also for developers. If you are interested in our jobs, please knock the door and contact us any time. We are always waiting for you here at SEGA.
こういうポリゴンを使って、スプラインメッシュでコースを作っています。コリジョンは見た目よりもかなり高い壁を作りましょう。でないとクルマが簡単に吹き飛んでコースアウトします。
