General observations about the inner workings of RPG Maker MV, plus tips and such for writing plugins

About RPG Maker MV in general

Since I've been doing some things other then "the game thing" for some time, I've decided to maybe write about it too. Of course, the game is not forgotten. I just struggle to find motivation to do more writing for it, even though the ideas are there. It will be done when the time is right, so let's proceed to the actual theme of this post: me digging through rpg maker mv sources and figuring things out.

Basically, I was given rpg maker mv to help out on a certain game that shall not be mentioned here, since nobody would understand either way. In particular, I was asked to help out on the "plugin" part. What this means: the whole engine is actually written in javascript, unlike the previous versions which, I believe, were written in something else and had ruby for a scripting language. Now since it's written in JS, the whole engine is basically an open box. You can modify anything, the whole thing. They give you the whole engine to mess around with. Except the editor, which is still a separate program, though the editor simply spits out JSON that the engine interprets (so you can also change how it interprets it too).

Now, the developers of rpg maker did something clever: instead of just giving you the engine and saying "yeah you can mess around with it" they added a system of "plugins". In essense, plugins are javascript files that are loaded after the main game. Since they are loaded afterwards, you can then override any part of the engine in your file without touching the actual engine code. Since javascript is prototype-based, simply changine the prototypes overrides the behaviour of the engine, while also leaving the original code intact if you want to use it later.

That said, unfortunately the creators of the engine abandoned it in favor of their new, slightly modified version of it, called rpg maker mz, a few years ago. And are selling mostly the same engine with somewhat improved editor. So no more updates to MV. Though since the whole engine code is available, that's not a big problem. One can make it whatever they want, it's just another 2D game engine that is somewhat steered towards RPG top-down games, but can do just about anything. Hell, you could make a custom editor for its json data system too, and replace their proprietary one.

About plugins and the state of the community

As mentioned before, plugins are the way to modify how things work without actually touching the engine code. An improtant part of plugins are parameters they can receive, and which can have different types to make the editor show different inputs for them. A good/only/official explanation of all the types is available here.

RPG Maker has official forums, and these do have a specific place for plugin discussion, tutorials and other things for each version of the engine. It seems that most people using the forums, except the few veterans who have been using the engine for years, don't have much idea about how plugins work, or about the inner workings of the engine. Since the editor is so good, a lot of the time they don't have to.

Though it is somewhat painful to see people giving clueless suggestions on the forums, with outdated ways of doing things in JS or simply not understanding how it works. Sometimes people say something is impossible to do, even though you have the whole engine source, so really anything can be done with enough effort.

One thing in particular I have noticed, is that nobody uses modern javascript constructs, though they have been available for a while in NW.js (the thing rpg maker mv uses to run on desktop, it's basically a webkit renderer with node for javascript). For example, evreybody seems to use the pre-es6 way of defining classes, simply because that's how they write it in the engine code, because the engine itself was made in pre-es6 times, I believe. So you see things like this (take from here), though admittedly the post is from 2017, ES6 was already available at the time (ES6 came out in 2015):

  function Window_MyWindow() {
      this.initialize.apply(this, arguments);
  }

  Window_MyWindow.prototype = Object.create(Window_Base.prototype);
  Window_MyWindow.prototype.constructor = Window_MyWindow;

  Window_MyWindow.prototype.initialize = function(x, y, width, height) {
      Window_Base.prototype.initialize.call(this, x, y, width, height);
  };

  Window_MyWindow.prototype.refresh = function() {
      this.drawText("This is a test message", 0, 0, this.contentsWidth(), this.lineHeight());
  };

instead of the easier and equivalent way of doing the same thing in ES6:

  class Window_MyWindow extends Window_Base {
      refresh() {
          this.drawText("This is a test message", 0, 0, this.contentsWidth(), this.lineHeight());
      }
  }

These are basically equivalent. Thankfully the whole engine is written in the same way that classes work in ES6, so everything actually just works with the new syntax.

People also still use var instead of let and wrap their things in function that is called immediately instead of just creating a scope with {}, which would just as well hide all names if you use let instead of var.

Of course, these observations are made only from the forums and some rare GitHub code one can find. Perhaps the people behind big games, knowledgable in javascript, use all these features, but we'll never know that. Still, the bad old code suggestions given on forums are somewhat disappointing, I hardly found anything useful there, and had more success just reading the engine code instead.

The peculiarities of event processing and drawing

Now, event processing is quite interesting in this engine. You might want to know this, if you plan on using the events provided via the editor in a custom scene that does not inherir or use the Scene_Map utilities. Maps are scenes which draw tilesets, have events, and are basically where the gameplay is. But there are other types of scenes too. The other important one is Scene_MenuBase, which has some utilities for implementing menus. So, if you want to have a custom menu and have events in it, or show images in it with the built-in commands, you'll have to work a bit for it, because Scene_MenuBase doesn't process events or draw sprites!

After some digging through the code I have figured out several things about how eventing and sprites work, and that's where some interesting differences from other engines surface. Usually, you either have a global manager that draws things and a global manager that processes events, or the objects themselves draw things and process events. But rpg maker mv seems to use both at the same time: to process events, you have to first create a Game_Interpreter and save it somewhere, then ask a global manager $gameTemp to create a reservation for the common event you want to run, and only then ask the interpreter to actually get the event from $gameTemp. After that, just call its update function every frame to make it process whatever the common event does.

The code looks something like this:

  class Test_MenuBase extends Scene_MenuBase {
      start() {
          // Reserve a common event received as a parameter to the plugin
          $gameTemp.reserveCommonEvent(Number(params["testEvent"]))

          // Create an interpreter
          this._interpreter = new Game_Interpreter()
          // Actually get the reserved event into the interpreter and clear the reservation
          this._interpreter.setupReservedCommonEvent()
      }

      update() {
          // Update the interpreter every frame
          this._interpreter.update()
      }
  }

Now, it will run the events, but that's not all of it.. It won't just draw pictures for you! Pictures also use this strange combined approach to get drawn. So, if you want to draw pictures, you have to add a Spriteset_Base to your scene. Which has a very annoying built-in behaviour of drawing a black background, that you might want to remove. The Spriteset basically takes all requests for drawing images with the built-in draw image command and draws them. You can also draw them manually by adding a Sprite_Picture to your scene with a specified picture number, but Spriteset_Base does this for you. The Map scene has both the spriteset and game interpreter in it, so it will process events and draw pictures, but other scenes don't. So if you add them to your scene, they will be able to do it. Quite obvious. You can also ask another manager to draw a picture instead of doing it from the editor, since all editor commands go through these managers. For drawing pictures, the manager used is $gameScreen. For example, call $gameScreen.showPicture to do the equivalent of what the editor event "draw picture" does.

Here is an example of a scene that can both draw pictures and process events:

  class Spriteset_Menu extends Spriteset_Base {
      createBaseSprite() {
          // Call the base function to create all the sprites first
          super.createBaseSprite()

          // Remove the black screen background. There's really no use for it in menus,
          // since it obscures the background
          this._baseSprite.removeChild(this._blackScreen)
      }
  }

  class Test_MenuBase extends Scene_MenuBase {
      start() {
          // Call the base function to do whatever it does first
          super.start()

          // Create the sprite set and add it to the scene
          let spriteset = new Spriteset_Menu()
          this.addChild(spriteset)

          // Fetch the common event that will possibly draw pictures
          $gameTemp.reserveCommonEvent(Number(params["testEvent"]))

          // Create the interpreter and get the event into it
          this._interpreter = new Game_Interpreter()
          this._interpreter.setupReservedCommonEvent()
      }

      update() {
          var active = this.isActive()

          // If you want for timed things to work, like picture movement, you have to update the $gameTimer manager too
          $gameTimer.update(active)
          // Update the game screen manager every frame, so it draws things
          $gameScreen.update()

          // Updates the interpreter, so it works on the events
          this._interpreter.update()

          // Call to the parent class' update so it does whatever it is supposed to do too
          super.update()
      }
  }

That's about it

That's about all I have managed to learn recently, for the particular task that I may be doing in the near future. I'll probably have to dig some more into other systems later, like the combat system. But I haven't done that yet. I imagine all the peculiarities apply to other systems as well, if they were designed by the same people.

I've hardly been doing anything recently, so I thought I might as well write out my thoughts on whatever it is I have figured out with RPG Maker MV, and hopefully this may help someone in the future.

Thank you for reading my blog post :)

Game thing progress #9, more editor things

I'm not dead, yet. The past few months have been rather boring in terms of things made, since I had to work on my diploma a lot. Hopefully I'm done with that soon and can come back to doing things I actually like. I'm actually using this game as my diploma project without any modification, so that's something.

I was told my game wasn't that impressive, that's true, of course, though the game is not the focus of the diploma, the engine is. So, I made an arkanoid clone in one evening just to make a completely different thing. To make it I had to extend the modding API slightly, but now the walking mode pretty much is all moddable too, so you can basically make your own new game using it, if you for some reason wish to do it. The documentation is basically non-existant on it, but you can look at the code in resources/mods/example and go from there, probably. Maybe I'll add some comments there to explain everything that is happening in the future.

Some fun editor things were implemented, like adding and selecting entitites that don't have a collider. Some more were added, since I wanted to make at least one room completely or at least mostly in editor. Which I did, but only one for now. Hopefully more to come soon.

Here's how the editor looks now, fancy buttons:

nil

The asset tab allows to add assets with the UI, too. It even has a fancy file selection window:

nil

When the asset that is in use is edited, the system tries its best to find and replaced all the usages of the old asset with a new one. Plus, if a Drawable component is added to an object, it initially has a placeholder black-and-pink texture which cannot be saved and serves as a temporary value until the actual texture is selected.

Other than that, the callback system has been adjusted to allow including type requirements for callbacks, so the editor UI can actually check these requirements and draw the appropriate editor. Basically, after you create a callback you call a special function and pass the type data into it, and it saves it in a special place for later usage. Here is an example of how it looks in code:

debug_components.declare_callback_args(
   M.interaction_callbacks.look,
   { prefab = "string" }
)

The callback accepts a single string argument in a map, the argument name is prefab. The UI uses that to create an editor that shows the appropriate table. The types can even be nested if some argument is a table in the aforementioned declaration. Or it can be a freeform table if its type is declared as table.

Most recently, I've tried to replace the SFML audio module with SDLmixer, which also made me add the entire SDL (though I've tried my best to disable unneeded things). The reason being that SDLmixer can have sound libraries linked statically into it and I can select only those I want to have - namely OGG. All the sounds are in OGG anyway but SFML also requires FLAC, for example. And it links to everythng dynamically which is a pain. I've been doing the SDL thing in a separate branch for now, hopefully I will merge it into master soon and the next release on will only require you to have alsa/pulse on linux and nothing additional on other operating systems.

Did you know that building SDL is actually a huge pain in the ass? Their CMake file sucks and for some reason SDLmixer's one has a check around everything that makes it only do something when building on android. I think I somewhat figured it out, but you can never know with these things.

Almost forgot.. I fixed an annoying bug I had with the TOML component saving. In TOML procesing code, the entity table itself is referred to as "self". So, the code tried to get the "self" node on every entity, even if one did not exist (such a node would have a path like "entities.entity name", with no component). If such a node does not exist in a file, but its source is retreived from code, it will point to some other node that actually exists which DOES NOT WORK and breaks things. So, now it only tries to use the "self" node in case there are no other nodes but the entity exists in the file: therefore it is guaranteed that the node will be pointing to an entity node with no components. Because of this bug, the saving was inconsistent: either the entity or the first component were selected at random, and when entity was selected things broke.

The shaders have been changed a bit, mostly moved to storing them in lua and C++ just sets them and then gets them from lua. With that, they can be edited more conveniently in the editor thing (which is basically the same as the callback or other table one, with types defined in C++ code and stored in lua).

That's about it, I think. Not much, but it's nice to write these things down sometimes. Hopefully someone actually reads them. And as usual, thank you for reading my blog post!

Link to the game on itch.io.

Game thing progress #8, actually mostly about editor stuff

Hello, Hello, all two (three?) of you reading. It's good to see you again. Once again it has been some time since the last post.

Many things have happened, mostly me also helping out on some other game, which also takes time (actually, I will get paid for it when I sort out paypal stuff so I'm not just wasting time, hopefully). But really, who cares about that, right?

You're probably here to read the progress on THE game, not some other game. It's been pretty slow. The current part will have more walking-mode content than all the previous ones together, and drawing and programming that stuff takes a lot of time, although I haven't touched it in a.. while. Mostly have been procrastinating or doing something else, after drawing three rooms and implementing two.

Since the release, I've implemented modding for the terminal-mode of the game. Mods can basically do everything normal lines can, but are loaded from the instance menu (you'll have to play the game if you want to know where the menu is). I've written an elaborate instruction on how to make them here, although this will only be relevant whenever the next release comes out. Nevertheless, I think it came out pretty well. When mods are loaded, they are provided a separate line environment than can only access its own lines and can't touch the main-story lines or other mods' lines. Mods can also load scripts, these are not vetted like the lines are, as I couldn't figure out a good way to do so. However, they're always available in namespace mod.<script name here>, that mod is literal, so you don't have to know the name of the mod. In general, the name of the mod never needs to be used in the API, so it's only for the players. When organized as such, mods can't interfere with each other.

For terminal mode, more smooth scrolling was implemented, which, unlike previously, actually looks pretty good and is better on the eyes than the text just jumping. Here's how it looks:

Some other improvements have been have been made to the room loading stuff, such as making passages fallback to previous days room files if one wasn't find in the current day, so rooms don't need to be repeated if they don't change. As part of work on the next day's walking part, touch-activated interactables were added, both always-working and one-time working (based on the activatable callback). These are used for main character commentary on some things, for example, and room transitions in some places.

A fade-in/out has been implemented and used whenever logically needed, like in transitioning from terminal to walking and transitions between rooms. It looks pretty good, and makes the player unable to walk in walking-mode while transitioning.

A way for coroutines to wait on something has been added, by making coroutines return special values from yield (in a way, this is similiar to how Unity Engine does it). This also has been used for providing coroutines with delta time. Previously it was wrongly passed as an argument, now when the coroutine resumes yield returns delta time for it.

A generic look-closer-at-thing system has been implemented, as there are now several things to look at that only show picture and text.

And now.. for the actually intersting part: editing TOML assets while in-game. For this, the pretty bad lua TOML parser has been replaced with a better C++ one and passed to lua. Additionally, all information on the position of the TOML node is saved, in C++ the whole parsed TOML file is saved for later, and for lua only the file and the path to the node is passed. When a component wants to have things saved, it provides a file and a path to the node and the toml-node-in-file is looked up and replaced.

A read-only editors are implemented for Transformable and Name, and for Collider an editor that actually does things is implemented (it also updates transformable in TOML source when needed).

As of now, new components can't be added, and neither can new entities, but the existing components can be edited. This includes checks for defaults, which are not written to the file if they are not overriden in the editor. Otherwise, if the default value is changed the key-value pair will be added to the TOML code.

The serialization-back system itself is implemented by finding the relevant place in the TOML file and replacing just this place with manual string manipulation on the C++ side. This is done so as to save the formatting and such of the original file, as it seems that no TOML libraries are able to do so and always re-format and re-arrange the file (very rude of them). As such, a system to only modify the data in relevant parts was made.

As for prefabs, currently editing anything using non-overriden prefab value will modify the prefab itself. This will probably have to be changed in the future somehow.

The editing will hopefully be expanded in the future and help with making new rooms, as it becomes quite tideous to maintain by hand. The hardest parts may be adding new entities and components, in particular figuring out where to put them in the file. Hopefully, I'll be able to work on that.

Here's how all of that looks:

Shown is the Collider editor. Currently editor clicks are only recognized for objects with colliders, other objects will probably have to somehow be made available otherwise as there isn't really an easy way of finding them in the world, although I do have some ideas for the future. Otherwise, objects with colliders can be dragged with mouse with CTRL+Click and clicking on an object selects it for editing, while double-clicking anywhere where there is no object deselects it.

Also, after my friend played the game (someone actually played it, crazy, huh?), he found out that it may so happen that non-english layout broke the game sometimes. That is because lua has string.char and utf8.char and the first one can't handle anything that is not ASCII. So I've replaced all mentions of string.char with utf8.char to hopefully fix the problem. That same friend found some typos and suggested changing the "description" color in the terminal part, as it didn't look well on black background and was hard-ish to read. So now the color is a bit more soft, as you may have noticed in the first video in the post. Another thing he noticed is an inconsistency where you didn't see something but it was mentioned later, that was fixed too, with the new text not mentioning the thing if you haven't seen it.

That was quite a lot of text now, wasn't it? Two videos, too! That's a record. I'm going back to making the editors better for now, probably. This seems to be promising. And also helping with that other game I mentioned. Happy new year, and thank you for reading my blog post!

Link to the game on itch.io.

Game thing progress #7, the long awaited second release and whatever has happened inbetween

Well, well, well.. Been a while, hasn't it? About four month to be exact. Sorry! To be fair, I mostly was not slacking off, you can look at the git history!

I have, I think, finished the story for day 3, which means the release should be coming very soon (upd: it's out). There is also some optional content that hopefully isn't too hard to find. Many things have happened, especially code-wise, next follow some interesting techincal highlights.

To implement a certain feature for the walking part of the game, there was a need to draw things over the shader-processed stuff the rest of the game has (that is, the darkness and the lights). For this, I have implemented a "layer" system, of which there is only one now (not counting the unnamed default layer). The overlay layer ensures that the object is drawn after everything else is drawn and without the room's shaders, which are now specified per-room and can have all kinds of parameters, there can even be multiple shaders! So anyway, the overlayed objects are drawn afterwards so they're not affected by shaders, and they still have the usual z level logic which allows to put them on top of each other on the overlay layer.

For some sprite related things, I have discovered (thanks to my dayjob, actually) the concept of slices in aseprite. Slices are basically rectangles on the picture that are saved in its json metadata file. I used this feature for that very same thing mentioned before. I mapped out two areas for putting text into them and then in code made text conform to these areas. Hopefully since the code is quite generic I will be able to use them for something else, too.

The code structure changed quite a bit, since there was need for dynamic entity instantiation. So, now it's easy to make new entities and the code that existed before takes advantage of the entity creation functions too. Overlall, the code moved quite a bit into modules.

The callback system was reworked a lot and I used that same concept in other places too: that is, callbacks now specify the module and the name of the function they want to call, instead of some hard-coded name. This makes things a lot easier to separate and is very good for modding.

This same concept was used for custom story lines: the lines which are too rarely used to have them built into the parser. These lines basically specify "it's a custom line, here's the module and the class, and here's the data to pass to it", and the parser does so. When implementing custom lines I came up with an idea to mark places where line names are referenced in their data with YAML type tags. Many people don't know they even exist but there were very useful in my case, since custom lines aren't directly processed by the parser it had no way to know which things to parse if the custom line wanted to jump somewhere in a different file. So when using a custom line if one wants to reference a line in it, it has to be written as !line-name name/goes/here. This makes the parser aware that this is indeed a line and it can pre-load the file and validate all the things at startup. Very useful and a lot better than what I initially did, which was just loading lines from lua at runtime. Needless to say, it didn't work very well and it didn't scale. An example custom line looks like this:

1:
  custom:
    class: { module: "terminal.select_line", class: "SelectLine" }
    data:
    - condition: |-
          return TerminalModule.state_variables.know_about_stuff
      next: !line-name know-about-stuff
    - next: !line-name dont-know

The few custom lines implemented and used currently are:

  • SelectLine, which selects from a set of possible ways to branch from some conditions and has a default condition to select if nothing matches. Sort of like switch~/~case
  • SaveLine and ReturnLine, these work together. The first one saves its "returnto" property and then jumps to "next". Afterwards, when one wants to go back, ReturnLine is used by specifying next_was to the value that was used in SaveLine and after loading the return line name from it it jump back to that line. Effectively, that means one can have, for example, a dialogue which can be started from anywhere with a SaveLine and then after it's finished execution is returned back to where the dialogue was called from, even though it doesn't not exactly where that happened. I used this to implement dialogues that will, most likely, be available throughout several points in the story.

Another custom line that exists is a special one-off menu, which I will not be discussing here because it's kind of spoiler-y, but it's not that interesting anyway.

The tags for the walking system objects were reworked a bit: now they created components automatically based on their name. Before, the tags where dispatched from hard-coded values to components created beforehand. In this context, a tag is a component (in ECS terms) which doesn't have any actual data but it used to mark an entity as something special.

A new puzzle was implemented for day 3's walking section, hopefully it's not too hard, since this time it's actually somewhat of a puzzle and I thing it can't really be just brute forced. It requires just a tiny bit of thinking, so to speak.

A testing system was implemented, for both C++ and lua's ECS. As of this moment, the coverage says about 50% but that's not really true. Not much is tested but the foundation is there. I have also moved back from moonscript to lua, because one can't really count coverage from moonscript, as no coverage tool exists for it. The only thing where moonscript still exists are the tests, because it's a lot more convenient to use there.

A full-blown templating engine was introduced in the terminal part, so answers could be easily changed depending on certain choices the player makes. Day 3 uses this functionality heavily, I hope it creates more choice for the player to enjoy, making this whole thing more of a game.

The walking mode also has received a debug menu, like the one used in terminal mode. For now it just shows state variables for the walking mode, but hopefully I'll add more useful features to it. Showing the whole ECS world structure would be cool to have.

Interaction system for the walking mode was changed a bit too, now interactables can overlap and if they do, the player will have a choice of which one they want to use: first pressing E to show the list and then the number of the thing they want to use.

A character parameter of font size was added, as it is used in the special custom menu line mentioned before. This menu has a header that writes text that's bigger than usual.

For the past month or so I didn't actually have to touch the code most of the time, as most of it was already written I just had to write the rest of the story. I tell you, it wasn't easy!

I also implemented an accurate way of counting words in the story, currently it shows 11789!

The past few weeks have been hard for me, due to an uneasy situtation in my country. I wasn't really able to concenrate on anything. After a bit I did actually start doing things again, even though the uneasy situation is far from over. Nevertheless, I think I can work on things now, so that's good.

The environment images sadly are still not really available. I didn't draw new ones, as it takes me a lot of time. I'm hardly an artist, and drawing even such small pictures takes a long time. I recenly tried to ask a certain person who in the past said I may commision him some art for the game when I'm ready for it, but it turned out after he published his game he left the industry as it was too much for him. That means, of course, I couldn't commision art from him. The search continues (though I'm not exactly trying really hard).

I also received a PinePhone recently. Let me tell you, it's not a good "phone" in the traditional sense as the software is very rough, young and made by the community, but it's a full-fledged computer, whic is great. It's so easy to cross-compile for: you just compile it with the usual desktop compiler for AArch64 and copy it to the phone, and that's it. Everything works. I spent the last few days tinkering with it before actually coming back to finish the story :)

Update from 06.09.2020: After writing this post I stumbled upon an interesting bug that took me a bit to solve, so I thought I'd write about it here too. When OutputLine is skipped too fast, the next line would be created too soon, before script_after is executed, that is, right after the interaction is processed (and scripts are executed afterwards in another function, which draws the lines). So, now the OutputLine explicitly says to wait until its script is executed before proceeding. This problem didn't seem to concern the other lines because they have more requirements than just "all text is output".

I have also implemented a way to scale a window down and up to half/double the size for smaller and bigger screens. It's not ideal, but it allows people to play even if their screen has a very small or high resolution.

I think that's most of the things.. Hopefully we shall meet again soon. And thank you for reading my blog post :)

Link to the game on itch.io.

Game thing progress #6, the first public release, automatic builds, and other things

Here I am again, with another post. Since the last time, the game actually saw its first public release on itch.io, as well as its source code and assets have been made public on github under the GNU GPL 3 license for code and the CC-BY-NC-SA 4 license for assets. Go check them out if you want, the code has quite a few intersesting places. And of course, you can now actually play the game for yourself :)

The whole story from the original .org file is now in the game and playable, which includes the first two days. I'll probably be working on expanding the story further soon.

Now, for the actual changes that happened since the last post.. When testing on Mac OS, I was told that the game actually didn't work, because the libraries it needed aren't usually there on user systems. So I had to learn how to make bundles for Mac OS. Luckily, CMake has a few nifty things to help with that, namely the fixup_bundle function from BundleUtilities which automatically finds the frameworks needed for the game and copies them into the bundle, together with the game binary. It's used like this:

install(CODE "
include(BundleUtilities)
fixup_bundle(\"${CMAKE_BINARY_DIR}/someone.app\" \"\" \"\")
" COMPONENT Runtime)

This means that when running make install, CMake will fix the bundle for me. Apparently, it has to be done on the installation step for some reason, which caused quite a bit of trouble because when you use add_subdirectory, the install target also installs all the things from those subdirectories. And nobody says how to fix that! I found an answer on some old CMake mailing list.. apparently if you add EXCLUDE_FROM_ALL as the last argument to add_subdirectory then it won't try installing things from subdirectories! So I added that to all the libraries that tried doing so.

Also in the process of building for MacOS, a friend of mine, who was basically the one who made it work on MacOS initially, discovered that lua from the lua github repository isn't actually meant for other people, but for lua developers, because it's missing a lot of things that are in the normal release tarball that is on the lua website. So, he replaced the lua submodule with a call to ExternalProject_Add that would download and build lua from that tarball. That made lua a lot easier to build on all systems, as there are now no hacks like compiling lua with gcc manually, it all "just works" if the correct platform is passed to the lua makefile.

Another small change that had to be made for the game to work with MacOS bundles was making it change the current directory to the one the binary is in, so it can find the resources directory. This means that the game can now be launched from anywhere, which is a good change for all systems, but crucial on MacOS because bundles run in a directory where the bundle is, not in the directory where the binary is.

Another interesting thing that happened is that I finally figured out why the game was segfaulting when closing, but only in release mode. The C++ static class members were to blame, and SFML in part. Apparently, SFML fonts implicitly depend on an OpenGL context, which, by the time the static destructors are run, is long gone! So making the static class members not static and passing them explicitly where needed fixed the segfaulting.

Another small quality of life change to the dialogue system format has been made: the answers of the player previously had to always specify the next line, even if it was a numbered line that was easy to figure out, which now it does.

Previously, the help text that said things like "Press 1 to continue" was shown inside the terminal with the same color as the character speaking, now this text is outside the terminal. It is also used in an all-new line type: text input line. This line has some "before" and "after" text, and between those the player can input some arbitary text with the keyboard (and the help line says they can do so)! For now, this is used to allow the player to input their name, and other characters will refer to them by that name. The name is saved in the same place all other variables are, and is inserted into text when the text is created by using Lua's string.gsub to replace things like <p> with the actual value of the variable p (which is the variable used for the player name throughout all of the story, but also it's the default name if the name input is skipped).

And now.. for some pictures! A debug menu has been added to simplify testing (for now, only for the terminal mode), the debug menu uses Dear ImGui and the imgui-sfml library to make it work with SFML easily.

Here is how line switching looks: you input the whole name and it switches the game to that line (very simple and very useful!):

And here's how the variable menu looks, allowing to see variable values (and even toggle boolean values, though this is an old screenshot where this isn't yet implemented):

nil

This menu is toggled by the tilde key, I left it in releases too, just in case.

The shaders for the walking part have been changed a bit, so now there's one shader with the computer screen light and one without. It's not the best solution probably, but it works for now.

The environment images have been disabled for the release because there aren't enough of them to cover everything yet, I'll bring them back when I have some more, but for now they're copied to the release but now shown

A lot of optimization work has also been done before release, the terminal part now works in more-or-less constant amount of memory, reusing as much as possible (not creating new text objects and recalculating them every frame is the biggest optimization probably). Many calculations that are used a lot have been cached, for example the size of the terminal frame is only recalculated when the window size changes, so it might even only get calculated once and then just be reused all the time.

Now a little bit about build automation: itch.io has a very usful tool called butler, which allows publishing on itch.io automatically and easily. I just download and unpack the archive on the autobuild machine, do ./butler push <release_dir> vaartis/someone:<channel> --userversion <version> and it uploads the difference between builds and makes them available under the new version, also hiding the old one. This works surprisingly well and I had basically no problems automating and using it, so all release builds (tagged as releases on github) are now also automatically published on itch.io. Since the initial release, I've made two bug fixing releases which were automatically pushed to itch using butler. Big props to people at itch.io for making such an easy to use tool.

And.. I guess that's all for now, even though the game isn't popular or anything judging from the itch.io statistics, that's really to be expected. I'll continue working and improving it. Thank you for reading my blog post!

Game thing progress #5, but also a bit of other things that's been happening

Hey, long time no see, as if anyone reads these posts.

I've been doing some things on and off for the past several months, including working on the game, a bit of working on another code thing, and playing games (haven't done that in a while..).

The other code thing is a somewhat ambicious and experimental (as in I have not done anything of sort before) project, where I try to implement a virtual machine and a parser for the C language in Common Lisp. Surely it can't be that hard, right? The answer is indeed, it's not exactly that hard, just tedious. I return to it sometimes when I want to work on something else, but it's been lying on the shelf for some time now. You can find the code on github1, but it's very not done and poorly documented. Because of course it is.

Now for the game, there are quite a few news, the first one is the game project files finally changing name to the actual name and not the old one left over from my previous attempt at writing the game in rust with a somewhat different plot.

Now that the first puzzle room is somewhat finished, I added a small jingle that plays when it's solved, composed it myself, too (putting a few notes together in LMMS).

The DrawableSprite component has been made into Drawable and can now handle any drawable, it's been made so I could also draw text with it. Currently, the only text drawn with it is the text that says [E] to <action>, action being some unique thing encoded in the object, so for a button it'd be "press the button".

The story has been revised a bit again, somewhat. After playing Syberia 3 a bit I felt like I read something I wrote. Unlike the first two games, it's written very amateurishly and I think I saw a clear problem with my own attempts at writing there. It tells too much, reiterates on every thing and makes everything very clear, so the reader has no room for thinking about what was said. This is why it feels unnatural, people don't talk like that, and realizing that I rewrote parts of the story to remove uneeded details.

There were lines of text in the story that required no choice or anything of sort, but just an acknowledgement (by pressing space to progress). These has been reworked to accept "1" instead of space, because as of now most answers only have one choice and therefore 1 is the most logical key to use for acknowledgement. That said, pressing 1 now also skips text "typing" and prints it all at once, this also applies to responses (though there any numbered answer key can be used). So now you can skip the text printing out if you want to go faster.

Also, the first character you meet has finally been given name. It is an obvious cut out of a different word, but I think it does feel like a genuine name, and it fits quite well.

The dialogue construction file format has also been improved, an option called dialogue_between has been implemented, it allows not specifying characters if their pharses are alternating (one talks after the other all the time), while also also allowing to change the character that is currently talking in the dialogue by explitly specifying the character name. If the character is in the dialogue_between list, the next line is assumed to be by the next character, and if the charcter is not in the list, then the tracking ignores that line and doesn't change the currently speaking character, thus allowing intermissions like player - description - other character, with only specifying that the description is a description, and the system understanding that the the description is not a part of the dialogue. That allowed to remove about 30 lines of character specifications that I already knew, but had to be provided explicitly. I'm kind of proud of this dialogue system by now, it's quite flexible.

More of the story has been added to the actual game from the draft file, soon enough I'll probably be ready to make the first public release, probably on itch.io, since it seems to be a very nice site for indie game developers, providing a lot of benefits, allowing easy communication, etc.

Since I'm somewhat getting ready for the first release, I set up automatic builds with GitHub Actions! It builds the game for Linux and Windows curently, though I hope to extend that for MacOS also (I don't have a mac, but I know someone who might be able to help me with that). While building for linux is easy, since that's the platform I've been using all the time and I engineered the build somewhat well, so that it takes just two commands to build the whole thing, Windows was a bit of a different story. On Windows, I had to use MSYS/MINGW, that took some time to figure out. Then, I learned that you can't build lua on windows that easily (and can't link to it statically at all), so instead of using their makefile, on Windows I do the following:

gcc -O2 -DLUA_BUILD_AS_DLL=1 *.c -c && mv lua.o lua.e && gcc -shared -o lua53.dll *.o
gcc -L. -llua53 lua.e -o lua.exe

This builds both lua-the-shared-library and lua-the-interpreter. The first one I link with and copy it to the release directory, the second one is used for luarocks, which has a different setup script on windows (it's called install.bat). This script gave me a few problems when automating, since the GitHub Actions thingy that sets up MSYS2 doesn't inherit the PATH variable by defult, while the real desktop MSYS2 does, and since install.bat, when it detects that it's run in mingw, assumes the existance of both unix and windows utitilies, it wouldn't work on the automatic builder, but worked fine on my machine! This was an easy fix (setting path-type to inherit in the action config), but it took some time to figure out.

Other than a few path (/ vs \) problems, there weren't much more trouble with building, mostly thanks to the fact that all the major dependencies are provided as submodules and are in my control and, in release mode, I compile everything statically, which makes distribution a fair bit easier.

For the last thing, I also also added a small delay between terminal line output, so they don't feel too fast.

Looking at it, not that much work has been done, I've been mostly slacking off, but I did have some stuff to handle IRL, though that is mostly over for the time being.

Well, that's probably my longest post yet. That's pretty nice, hope you enjoyed it, and hopefully we'll meet again soon, thank you for reading my blog post!

UPD 25.09.2020: Link to the game on itch.io.

Game thing progress #4

Getting straight to the point, since the last time a system for rooms to use other rooms as "prefabs" has been implemented, they can remove entities and resources they don't need, put new ones and override old ones. This has been used for a room that looks almost identical to the first one, except it has a door instead of a button, so many things can be copied over (like wall colliders and background).

While implementing this I noticed that colliders don't handle Tranformable scaling well, so I made a simple thing to adjust their position for this. Now, a door that has a mirrored texture has its collider put where it should be and not on the right.

The interactable objects also now use collisions to detect the player interacting with them instead of just their sizes, this allows for invisible interactable objects, for example.

The asset system has been reworked: now all the asset names are in one place instead of in the objects that use them. This make it easier to actually reuse things. Also, the asset table in lua now only stores the things that can actually be shared (textures and sound buffers instead of sprites and sounds) and components create their own instances of classes that use these shared assets. When the walking module is first loaded, it reads the assets file and puts everything in it into the known_assets table, which, through the __index metamethod on the assets table, is looked up when the asset cannot be found for a path to the asset. It is then loaded and put into the assets table for later usage.

A very simple and basic guessing "puzzle" was implemented for the ending of the first day. It is in the second room and involves three buttons interacting with each other. Along with that, I fixed a bug that was caused by not resetting the physics world when changing rooms (but resetting the ECS world).

Then, I separated the "can this thing be interacted with" and "what does it do" into two different fields in the interactable component. This will probably be used in the future to show a popup when something can be interacted with. It was added to all the objects that can have some condition where the object becomes non-interactable.

A small timeout has been added to object interactions in attempt to prevent interacting with multiple objects at once.

Getting back to the terminal text part, I implemented switching to different rooms from the terminal. In the process, I fixed YAML cutting off newlines (because I used > instead of |). This was a problem in some cases where lua requies newlines, but in some cases it wasn't noticable at all, so I didn't even discover it at first until a certain point in the script.

And that's it for now.. Thank you for reading my blog post!

UPD 25.09.2020: Link to the game on itch.io.

Game thing progress #3

Since the last post, everything except the terminal part has been rewritten in moonscript. I'm still not sure if I like it that much better, but I like to think that the time spent dealing with the lua module linking hasn't gone to waste.

A simple change has been made in the walking part, to make it a bit longer, requiring the player to go back to the computer, where they previously were at the start, and then interacting with it to get back to the terminal, instead of just going there when the "puzzle" has been solved.

A simple prefab system has been implemented. It allows separating entities into different files and loading them from other scenes. This way, the player entity can be recreated in the new room easily.

A beginning of a second room, which will contain a simple puzzle, has been drawn and inserted into the game, along with a passage sprite to separate the rooms. When the player interacts with the passage, the ECS resets and loads the new room's contents. The passage is, in turn, implemented using the same mechanism the push button used, except it's been generalized for usage with anything interactable.

A simple collision system, using bump.lua, has been implemented, which replaced the "just compare sprite bounds" system. This allows entities without sprites to have collisions too, invisible walls (or walls that imitate the level geometry) can be made this way. If the entity does have a sprite though, the collider component can be told to use the sprite size as its bounds, or, as with those without the sprites, it can be told to use constant sized bounds defined in the entity description. The position for the boundaries is taken from the entity's transformable component (which is basically SFML's Transformable). If the entity has a sprite, then the sprite as used as a transformable, and if it does not, a new transformable is created and used.

The above exposed a bug with render Z sorting: before, everything was drawable, so this wasn't a problem, but introduction of things that are not to be drawn also introduced gaps into the entity array of the drawing system. This means that virtually all built-in things that work with arrays do not work anymore. Particularly, sorting either silently fails or passes nil to the sorting function, because it assumes there are no gaps. It took me about a day and a trip to the #lua channel on IRC to figure that out. Now, there's a separate array of entities to be drawn, sorted and without gaps, which is updated when entities are added or removed.

Synchronizing collisions betwen the physics world and the actual trasnformables and moving things accoring to those collisions also turned out to be a non-trivial thing to do. I implemented a debug overlay that draws red boxes around colliders, this overlay will probably be easily togglable in the future. The positions of the objects are now synchronized with the physics world in the following way:

  • If the entity wasn't in the world before, put it in there with the position of the transformable minus the origin/pivot point
  • If the entity was there already, put the position from the world plus the origin/pivot point, and update the size of the entity in the physics world from the sprite

And that's how things have been in the past few days, I think that's enough for this post, so I'll end it here. Thank you for reading my blog post :)

UPD 25.09.2020: Link to the game on itch.io.

Game thing progress #2

Since the last post, some more of the story text got ported to a format that the game understands. The entire first day is now there, and, maybe, it will stay there as it is, unless i decide to change the beginning details again. This day includes a feature that I implemented a long time ago but didn't have any chance to use, which is showing answers to questions depending on some state. This state is actually determined from a lua script in the story file. Here's how one of them looks like:

return not TerminalModule.state_variables.narrator_house.hub.living_room

This "script" is put into the "condition" part of the reply and the reply is therefore only shown if, in this particular case, the living room has not been seen yet. I'm thinking about maybe simplifying hubs like these in some way, but I'm not sure about it yet.

Another thing that's been done is that for the "walking" part of the game, all the components for the entities and the assets to use are now written out in a TOML file (because that's a simple format and there's a native parser for lua), and are loaded from that file instead of being hard-coded. Here's how that file look like right now:

[assets.sprites]
computer_room = "resources/sprites/room/room.png"
button = "resources/sprites/room/button/button.png"
mainchar = "resources/sprites/mainchar/mainchar.png"

[assets.sounds]
footstep = "resources/sounds/footstep.ogg"

# Background

[entities.background.drawable_sprite]
sprite_asset = "computer_room"
z = 0

# Button

[entities.button.drawable_sprite]
sprite_asset = "button"
z = 1
position = [1020, 672]

[entities.button.animation]
sheet = "resources/sprites/room/button/"
playing = false
playable = false

[entities.button.button]
initial_state = "disabled"
state_map = { disabled = 1, enabled = 2 }
callback_name = "switch_to_terminal"

# Player

[entities.player.drawable_sprite]
sprite_asset = "mainchar"
z = 2
position = [420, 820]
origin = [44, 176]

[entities.player.animation]
sheet = "resources/sprites/mainchar/"

[entities.player.player_movement]
footstep_sound_asset = "footstep"

All of the assets are loaded and stored first, and then components reference them by name. At component creation time, references to assets are taken from the storage and put into the components. This allows them to never be GCd, which is a good thing since dependencies between values passed from C++ like textures are not tracked by the lua refcounter.

Another change worth noting is porting most of the code to moonscript, because it's quite a bit better as a language (except maybe the indentation syntax, the features are nice). That was a pain to set up, because i've always been linking lua statically to my executable, but lua modules expect that the dynamic symbols of the lua library can be looked up. Didn't know about that for a while, but when i figured that out, it was as easy as adding -Wl,--export-dynamic to the linker arguments to expose those statically linked library symbols to the modules. After that there was a bit of a fight with luarocks, that ended up in moving the modules from build.modules to build.install.lua, as it was trying to build .moon files as if they were C sources. For some reason, sources in install don't have that problem.

So, that's what I was up to in the last few days, I'm hoping to maybe continue rewriting things in moonscript and then work on the second room in the walking part of the game.

Thank you for reading my blog post :)

UPD 25.09.2020: Link to the game on itch.io.

Game thing, initial story

I've had an idea of making a game with a story inspired with several things I loved a lot for a while. So I started actually doing it about a year ago, in rust, because that's the "popular thing to use". I wasn't sure about the plot at the time, just focused on the engine, it had a big field for the story text and a few buttons with answers at the bottom. It was in that state for a while, I used a different story as an possible intro, since I wasn't sure about where to start. And then I kinda burned out on the whole thing and forgot about it for half a year or something like that.

I remembered about it when playing another game, not related to the original inspiration, but also with a good plot. That day I started thinking really hard about what to do and decided to scrap the previous story prologue and start a new story unrelated to it. I was super determined initially and spent a few evenings writing the basic, very rough prologue into an Org file. It was a few thousand words when I finished that session, and I thought "Well, I guess that's pretty good for my first serious writing attempt, right?", and went to sleep, because it was super late already :)

I spent a few days adding things to it after that and then tried changing the engine to suit it better, since it was more like a terminal with inputs now, and not a screen with a buttons. In the middle of that process I got tired of fighting rust over simple things and decided to rewrite the whole thing in C++ because "I know what I'm doing, just let me do the thing", so that took a few days too. But not that long, because I was just porting a few simple algorithms and using the same graphics library1 I was using in Rust except in C++ it was natively available.

After finishing porting the needed parts (only the terminal one, basically), I worked on the other planned part of the game, where the character is supposed to actually walk and interact with things and not just read a lot of text and respond to it. Doing that in C++ sure was a pain. So I thought maybe I could do it in Lua, and turns out there's a great library full of magic2 that greatly simplifies using lua with C++. After some painful CMake-ing, I was able to connect luarocks and CMake to pull all the lua dependencies and put them into the _buid/resources directory, where the game could find them at runtime. The next few days were spent porting things from C++ to Lua. I thought about using ECS for the terminal initially, but turns out it doesn't work very well in that particular case, so I resorted to just having two functions called from the C++ part that were called every frame to draw and update things. When the porting was done for the terminal, though, it looked a lot better than C++ version and I wasn't that much ashamed of it.

After that came a bunch more writing. I set up a goal of finishing at least two days worth of text (two game days). That took a lot of time, and I'm still revising what was done there, but it's a good amount of text for now, about 8k words. It's not well written, it is my first big writing experiment after all, but I'm somewhat proud of it. I had some people read it and give me suggestions on what I should improve about my writing, so let's say that is in the works.

At some point, I decided to draw a few small pictures to explain surroundings a bit. They are small because I'm not very good at drawing pixelart and there's not much space on the screen for them anyway. I drew the first two for now, but more will come later, I already have them planned in my head.

With the first few days finished, I thought I'd also port the "walking" (that's how it's called in code) part of the game ported to lua too. Now that part did actually work well with ECS, so I used a ECS library for lua3 to do everything in the first walking scene, using some global values provided from the C++ part (like the drawing target). I'm not sure about how it performs, but it was once again better than what I had with C++. I added a lua coroutine wrapper thing that would simplfy working with them and implemented fade-in/out transitions to switch between the terminal scenes and walking scenes, that took about a day, but coroutines will surely be of use in the future. And all of that is built as simply as just typing make, thanks to past me doing all the awful CMake writing.

As of now, I'm continuing to work on the story, refining and rewriting it, making it seem more natural and belivable. Stay tuned for some more progress news, hopefully.

UPD 25.09.2020: Link to the game on itch.io.