Archive for November, 2012


Today I did a video

About time I made one of these.


Uneven playing

We need to get people to play more during the days. This curve showing the last 24 hours is rediculous


Version 1.41 for iOS available now

Finally after almost 7 days Apple has approved v1.41 for iOS so let’s go update. RummyFight is so much more fun when the notifications work.

In the meantime while waiting for the iOS version release I have worked some on the Android version too and finally managed to implement a feature that has been requested for quite a while.

I’ll attach a picture and see if you can spot the change.

Of course there are other things in this coming Android version too but I’ll wait to tell them until it’s time for the release… which hopefully will be tonight or tomorrow.


First update of RF for iOS

It never fails. Regardless of how careful you are there are always things discovered once you have released an application and RummyFight is no exception to this. A few minutes ago we therefore uploaded v1.41 for iOS to Apple including some bugfixes. Nothing major except a fix that makes notifications work. Yes, unfortunately they doesn’t work in v1.40 due to the fact that Apple doesn’t provide a test system 100% identical to production so it’s no wonder that Notifications was the one to fail. Anyway, I have found the problem, corrected it and tested it and it seems to work now.

Other things included in the release are some minor visual updates and a few other fixes. Nothing huge. I mean, the most serious one is that you couldn’t put two jokers next to each other when forming a 3 tile group containing either a 1, 2, 12 or 13. Like 12,X,X. The reason for this is that one of the routines checking for illegal melds got confused when treating it like a Run since it would mean the Run is 12,13,14 and 14 is not a valid tile.

But I mean, how often does this happen?

So now we will see how fast Apple is handling updates. The initial release took almost 8 days. Let’s hope a simple update gets approved faster.


We are live!!!

Once the review process finally started, the app was approved in 2 hours without any objections or complaints at all from Apple.


Significally faster notifications

Lately it’s been evident that the notification pusher isn’t fast enough. Every 60 seconds it is activated and sends out notifications to players who is in turn, has an invite, a new chat message etc. The problem is that each request to Google’s notification service takes about 0.5 seconds and it means that the capacity only is about 300 notifications per minute – not enough anymore during peak hours where there’s a need to send out more.

So I have now rewritten parts of RummiPusher to be multithreaded. This means that we now can do multiple requests at the same time. Initially we are using 5 threads which means that in the same time we could send one message before, we can now send five. The number of threads can easily be adjusted at needs but at the moment 5 is well enough to be able to send all notifications within the minute at hand.

Sometimes multiple threads is a blessing… sometimes it can be a real nightmare to code since multiple threads also means that two pieces of code sometimes want to access the same object at the same time, leading to uncertainty of what value this object should have.

A simple example is this. Assume that we have a simple multi threaded application where each thread wants to increase a counter by one. If that counter is 10 and is accessed by a thread, it turns to 11, right? Assume that two threads want to increase this value at the very same time. Then both of them “see” that the value is 10 and increase it by one. The result is that the counter is 11 when it should be 12.

This is called a “race condition” and should be avoided like the plague. In such a thing like RummiPusher this is not so serious. I mean, no one would lose their life if a notification is not sent. There’s another thing if this would happen in the app – which it has. Up until v1.38 of RummyFight there was a race condition during the re-play of other player’s melds if you are holding down FastForward. As you have seen there are a lot of tiles moving at the same time then (multiple threads) and at rare cases a tile could be misplaced or “forgotten” by the re-play routine due to the fact that it (sometimes) failed to access a tile if it tried to move it WHILE it was already being moved elsewhere.

But that was fixed in v1.38 and later versions so that’s nothing we need to worry about anymore but it shows the danger with multiple threads and I can tell you it’s a real bitch to debug since it is almost impossible to simulate the situations where this can happen.

Ah well, I do wish however that Apple could show some multi threading in their app approval process since it’s now been 7 days and they haven’t still even looked at RummyFight for iPhone.


The daily curve

One definite challenge with RummyFight is to provide enough speeds of the servers to allow fast response times for the app. Every time there’s a fight opened, it must be fetched from the server. Every time a meld is done, it must be stored. Every time the lobby refreshes, it must provide accurate information etc etc. This is not as easy as it sounds since there are lots of players playing at the same time. At peak hours there are more than 600 database requests made per second and that’s quite a lot. I’m actually a bit surprised (and proud) that most database operations take less than 0.100 seconds to do.

Peak hours… yes. We do have peak hours.

It’s actually quite funny since it’s quite predictable. The most playing is made on Sunday afternoons and evenings (european time). The highest peak is also on late Sunday evening. Then everything goes down during the night (the american players obviously play less than europeans). Then during the day it’s a constant increase up until the evening followed by a big drop at about 11pm swedish time. This is probably due to the fact that RummyFight is very popular in the Netherlands and 11pm Swedish time is about 12pm dutch time. Time to go to bed, really.

But one challenge about all this is to dimension the servers to handle the peak hours, not the regular hours… and as you can see below, there’s quite a big difference betweeen days and evenings.

By the way, have you noticed that this curve is also a part if the “blue world-theme” we are using in the app? 😉

Regarding iPhone-RummyFight: Still no reaction from Apple. Almost 6 days now without them even looking at the app. This makes me disappointed but unfortunately not surprised. I’ve been warned about Apple being the slowest player on the market and sadly this also seems to be correct. Hopefully they can make it up with quality instead.


Ups and downs

Regular players know there has been some ups and downs on some of the systems of RummyFight lately. Especially the chat service and notification push. This is the results of us changing things, optimizing stuff and trying out new ideas. Unfortunately this has the drawback of the ideas sometimes not working or has to be carefully added for a couple of fights at the time. I can’t say the systems have been down a lot but it could be annoying enough to have the chat disabled for 5 minutes if there is something you would like to say to someone.

We apologize for this and can comfort you that we are done with the fiddling this time around and that the systems should be back to 100% now again. The results of our efforts have been good though. We can now tweak out some more percents out of the system and also some increased speeds. And this is good 😉

Regarding the iPhone release of RummyFight nothing has happened. It’s still in “Waiting for review”-status meaning that Apple hasn’t even touched it yet even though it’s been almost 5 days.

They really have to do something about this initial waiting time. I have no problem with them taking their time reviewing new apps but not having the resources to even start reviewing new submissions is just plain bad.



We are well aware of the possibility of a dramatical increase of usage once the iPhone users start arriving and since we love our very rapid response times of the servers today we would really like to keep it that way even though the traffic will be heavier. I am an avid player of other asynchronous mobile apps like wordfeud, ruzzle, nummi and similar and am very proud that none of them manage to deliver the same response speeds as RummyFight.

But this will probably not be the case in the future unless there are some optimizations and tricks made on our servers. We will therefor try out some ideas in the coming days to see what impact they will have on the server speeds. There are still lots that can be done especially in the database cache department but also in the communications between servers. Today RummyFight is using 2 servers for the main game but none of them are any monster and I like it that way (one of them is actually very small). Anyone can make a huge server move fast but the challenge is to tweak every little percentage out of smaller machines since it will make a huge impact the day we switch to larger ones.

We are also experimenting with an interresting idea since last night. About 1000 fights are currently outsourced to simple web hotel and the idea is to find out how well this works. If it’s a success it means that we will be able to use any computer out there capable of running .NET and MySQL as a fight server. It can be located anywhere in the world and it won’t matter. It will almost be like a cloud setup and that can give us almost unlimited computer power.

But first thing’s first. I do believe we can still tweak our in house servers to be able to handle at least twice the load today.

Still no word from Apple regarding our submission of RummyFight by the way. It’s still in “waiting for review”-status.


It’s submitted!!!

Submitted RummyFight to App Store about an hour ago. Now we’ll just have to wait and see… and hope.

The app will be available for iPhone, iPod touch and iPad in stretched mode. Of course it will not look super duper on an iPad since it will be zoomed there but at least it will look great on an iPhone and iPod. Don’t you agree?