Since I closed business in early 2019, the site was well deserving a new, more personal touch (hi Richard!). So after spending a few hours finding an acceptable WordPress theme, there it is. I’m not sure I succeeded though. Hopefully there will be more articles from now on.
Why you shouldn’t use Google Chrome
Until now I used Chrome with a few settings tweak like disabling sending usage statistics, uBlock origin, no crash reports and so on.
Then, when checking why one my newly installed WordPress themes was using some Google fonts, I came upon Chrome sending this with a GET request:
x-client-data: CJW2yQEIpLbJAQjEtskBCKmdygEI67jKAQisx8oBCPbHygEItMvKAQjc1coBCJeaywEYisHKAQ==If you check with Chrome’s own Network analyser tool, it’ll automatically explain what it is:
message ClientVariations {
// Active client experiment variation IDs.
repeated int32 variation_id = [3300117, 3300132, 3300164, 3313321, 3316843, 3318700, 3318774, 3319220, 3320540, 3329303];
// Active client experiment variation IDs that trigger server-side behavior.
repeated int32 trigger_variation_id = [3317898];
}If you type about:version into Chrome’s URL bar, it’ll display something like:
Which is a long list of “variations”. Google claims this is used to allow rolling out features from their servers to only a small subsets of users (so they do need a kind of unique ID for that). So for example if you’re watching Youtube, you’d get the new UI refresh only if your ID is included.
Can that ID be used to track you? Yes it can. And if it can, you can be 99% sure that Google is doing it.
You cannot remove that feature. The suggested workarounds are to disable “send usage statistics” which restrain that ID to 13-bits (which, with your IP address is still more than enough to track you) or run Chrome with some obscure flag that makes it generate a new ID on startup, which is useless if your browser is running all the time.
As for me, I’m switching to a better browser.
Archiving YouTube
Lately, YouTube has been on the hunt to take down informative videos. It’s often hard to know the exact reason and it seems even a few users signaling a video will make it vanish.
Most people use the Favorite option of YouTube to save interesting videos, but once a video is removed, this is what shows up on the playlist:
No title, no description, nothing. YouTube is effectively erasing every historical trace of the video. You can’t even decide if you agree with their policy. It’s not like they give you a choice anyway.
So what to do? The solution would be to download the video and store it somewhere. After all, extra storage is pretty cheap nowadays, but YouTube offers no download button.
Enters 4K Video Downloader
This tool allows to grab any YouTube video quickly.
It can even save entire playlists, like for example your Favorite playlist in YouTube.
And best of all, it’s free. So the next time YouTube deletes some videos you won’t care. You already have them.
Download here.
Reliable monitoring with Logitech Gaming Software and Arx Control
Arx Control is a software for Android that allows you to monitor your PC’s hardware temperature and resource usage in a nice way. With a small external display provided by any cheap Android tablet or smartphone, you can effectively monitor the GPU temperature and fan, CPU temperature and thread usage and memory use without having to change anything on your screen. Think of it as a small external display without the inconvenience (room and multimonitor setup).
Unfortunately it suffers from a major drawback: it works using the wifi connection of the device.
This would actually be usable if the app had a system to automatically reconnect but in practice you waste time killing and relaunching it, which defeats the purpose of convenience.
But there’s a way to fix it: we’ll simply use the USB connection by tunneling a TCP socket into it.
First, the tablet/mobile must be connected to your computer with an USB connection (which has the added benefit of charging the device). You also must have ADB installed.
Then this is how to proceed, first, make sure your device is NOT paired by wifi, if it is, remove the authorization from Logitech Gaming Service in Settings / Arx Control on your PC.
It would be simple to just turn off wifi on the device wouldn’t it?
The app wants wifi so just enable it and let it fail.
Now you can disable wifi on your device. I recommend it if you don’t need network connectivity for anything else as it’ll make it easier to charge the battery on the USB port.
Open a Windows shell and type the following:
adb forward tcp:54644 tcp:54644
Next go into Logitech Gaming Software, click on the settings wheel and go into Arx Control. Enter 127.0.0.1 below and click Connect.
If everything went well, you’ll be congratulated by the following:
Now you can enjoy a reliable monitoring.
Unfortunately the whole procedure has to be made again if the PC is rebooted. But it’s still a clear win compared to an unstable wifi link.
How to fix UPNP IGD not working on ASUS RT-AC87U
UPNP IGD is a system used to allow clients behind a NAT to be reachable for incoming connections. This is very useful for all kinds of P2P protocols and is a standard features of most Internet routers.
But it wasn’t working right on my RT-AC87U. It would work after a fresh router reboot or when a connection was made but after a dozen of minutes, no application was able to add port forwarding entries.
Fortunately, I used to do contractual work involving protocol analysis so I fired up Wireshark to see what’s going on.
UPNP uses multicasting to advertise itself. So I just simulated a client using the UPNP Tools. I could see how Windows would send an IGMPv3 Membership Report but then it would never see an IGMP Membership Query from the router.
On another machine hooked to an ethernet port, I would see them, though.
After a while, it seems this is because the router doesn’t let IGMP pass through the WiFi interface. I’m not sure why this is the case but it might be because having multicast packets go through WiFi is usually hell, as described in a previous post of mine.
The fix isn’t very intuitive because you have to go into the IPTV settings of the router, even though it doesn’t seem related with IPTV at first. So go to Advanced Settings / LAN / IPTV.
If you use IPTV, make sure each wireless interface has Enable IGMP Snooping ticked, to avoid multicast flood.
And that’s it. UPNP should now work perfectly! I’m a bit surprised there’s like zero information about that even on ASUS forums.
Life without Google
Update: Qwant has admitted censoring search result worldwide (ie. any country) upon simple request from French organisations without court orders, which is why I no longer recommend Qwant. Use duckduckgo instead.
A friend who’s a fan of real time strategy games told me that a game he was waiting for, Syrian Warfare, was removed from Steam by Valve, without apparent reasons.
I wanted to know more about this, so I headed up to Google and typed:
syrian warfare valve banned
But the result annoyed me.
What has Donald Trump to do with the removal of a video game? Absolutely nothing. Same for the UN story. Of course, one could think Google just tried to be smart and enhance the results but here’s what competitors give:
I know it’s no secret that some Google executives are pro-Hillary but they’re going too far. Funnily enough, another friend who’s pro-Hillary is tired of seeing anti-Trump results in his Google searches.
Wait a minute. Notice these competitor’s search results? If they’re better than Google for that particular case. Could they be just as good for all other searches?
The CEO of Hacker At Work suggested I use Qwant and so I did. I just switched my browser’s search function to it and used it for all my searches during 2 weeks.
Turns out I was wrong. Qwant is not as good as Google, it’s better. Gone are the duplicate results when I search for some Android bug or the 3 copycat sites with the same response from stackoverflow. The image search is fast and I don’t have to constantly fiddle with the Tools/Verbatim/Past week options of Google.
Maybe with machine learning, higher bandwidth and cheaper storage, we’re turning to a point where Google can finally have some real competition. While Google, which started with the simple (but novel at the time) Page Rank algorithm, were fighting spammers and SEO blackhats trying to game their algorithms, others were using new technology which are really effective today.
I now happily use Qwant and don’t plan to go back to Google’s politically filled results.
(And Syrian Warfare is a great game, too)
It’s all about maintenance
Programming is like sex: one mistake and you have to support it for the rest of your life.
— Michael Sinz
How would I guess, back when writing the first version of I’m sleeping, that I would constantly need to come back to it with each release of a major Android version?
Even though Android has a mechanism which is supposed to eliminate such kind of burden, it wouldn’t work in this particular app’s case.
The purpose of the app is simple: wake at a precise time, twice per day to set or unset the volume of the phone’s ringer.
Easy enough! Simply use the framework’s AlarmManager:
alarmManager.set()
No, wait! Starting from Android KitKat (4.4), set() doesn’t guarantee precise timing anymore because of battery saving reasons. One has to use a new call:
alarmManager.setExact()
No, wait! In Android Marshmallow (6.0), setExact() doesn’t work when the device is in idle mode (ie. left alone on a table without moving), which means you’ll miss alarms. One has to use another new call:
alarmManager.setExactAndAllowWhileIdle()
The annoying thing is that each of these API changes would simply break my app if I wasn’t closely monitoring the framework’s evolution. I really hope there’s won’t be a setExactAndAllowWhileIdleAndDeepFreeze() or so call in a future revision of the framework.
This is a good example of why perfectly working and stable code still needs to be maintained because of all the changes around it (OS, drivers and libraries).
How to make your Galaxy Note 4 fast
Update: the following was tested for N910FXXU1COH4 and N910FXXU1COI3. For N910FXXU1COJ3 Samsung added an “AppFreezer” system. But it still kills background processes.
The Galaxy Note 4 is an amazing piece of hardware. Recently, it was updated to Android 5.1.1 which solved many performance problems the previous Lollipop (5.0.1) release had.
Unfortunately, the phone still feels sluggish. When you go back to an app that was used 5 minutes ago, you can see how it recreates the app from scratch, there’s a delay with many actions and it feels like an old phone. If you compare it with ie. a Nexus 5, the latter feels faster to use, yet it’s older, has a weaker CPU and less memory.
But worse! With the 5.1.1 release on the Galaxy Note 4 some apps will behave badly. For example you won’t get incoming VoIP calls, missing messages, missing notifications on Android Wear and so on.
People often blame TouchWiz (the Samsung framework modifications). So I decided to have a look at their framework.
Samsung indeed does change the process management part of their device. It seems this was added during the early Android versions (1.x, 2.x) and most of it is not necessary anymore. The default memory management of Android was greatly enhanced since Android Honeycomb (3.x) and even more since ART in Lollipop.
Basically what Samsung does is kill apps and services in the background very frequently. I’m really puzzled of what’s the point in it. I suspect the Samsung engineer responsible for memory management is pictured below:
And there’s a lot of memory management systems in their framework. It seems that when they noticed memory problems, they just wrote a new one!
Fortunately, there’s a way to solve it:
- root your device
- edit system/build.prop with the following:
Find the following line and switch it to false, like this:
sys.config.samp_spcm_enable=false
This disables the service killer. Now your services will run reliably. Let’s add the following lines to disable their memory management:
sys.config.samp_enable=false ro.config.fha_enable=true ro.sys.fw.use_trim_settings=false
The first line disables their Smart Adjust Manager which is a system that tries to rate tasks, do some statistics and kill the ones that seem to take most memory. It seems like a good idea at first but unfortunately it kills tasks you use the most as well and has absolutely no place on a system with 3GB of memory.
The second line enables a simpler memory management system (in fact, it disables most of the old memory management stuff, the ones that you configure with dha/sha lines). It’s very close to the AOSP one.
Finally, the TRIM system, as its name implies, removes cached apps. It seems it was designed for devices with 1GB of memory or less. Why it’s enabled on a Galaxy Note 4 is beyond human understanding.
With these lines, the device feels just as fast and smooth as an AOSP device. Your battery will also lasts longer during active use because the device won’t spend time killing and recreating tasks all the time. Multitasking is a joy again.
This proves that TouchWiz is not bloated but it’s just their braindead memory management.
Note: these modifications likely work with other Galaxy devices but I don’t have the time to reverse engineer all of them. If someone wants to try (Galaxy S6, S6 Edge, Note 5) feel free to report your experience in the comments below.
The Android battery problem
Whoever owns an Android device might wonder why it is that, most of the time, the device barrely makes it through the day, even with moderate usage.
After many experiments, I finally found the culprit: Google Play Services and its location feature.
There are 3 settings in the location mode. Let’s go over them:
Device only: this setting is the most simple one (as long as the definition of simplicity does not take into account the math needed to derive a position from satellite beacons). It is the same system that a standalone GPS will use. Lock on to several satellites, download their ephemeris data and compute the position from that. The advantage is that there’s no network needed, just a clear view of the satellites. The disadvantages is that it can take 10-30 seconds and it doesn’t work so well indoors.
Battery saving: this mode will scan for a list of current WiFi accesspoints and the phone tower the phone is connected to then compare them with a (possibly cached) database from Google. Since Google knows the position from the data, it can give back a position. The advantage of this mode is that it’s very fast. The disadvantage is that it requires network coverage and communication with Google if the data is not cached.
High accuracy: this mode combines both previous modes. It scans for WiFi and phone towers, gets the data from Google but then, it uses the GPS to confirm that data, or, if Google had no match for it, it creates that new data for Google.
Wardriving for the masses
What happens when using High accuracy mode is akin to wardriving around with a laptop and a GPS and submitting the data to a central server (Google). This uses a lot of battery. Not only is the GPS often used but there’s plenty of network communication with Google and WiFi scanning. The device is also often woken up to perform a fix.
So which is the mode which uses the least battery power? It’s Device only. Just try it for a few days and you’ll see. The battery saving mode’s name is just plain bad naming. It will still perform scanning, communication with Google, and it won’t be very accurate in case there’s no access point around.
My Nexus 5 switched from not even a day of battery (high accuracy) to 3 days (device only).
Mail server fun
SMTP means Simple Mail Transfert Protocol.
As usual, when there’s the word simple somewhere in the name of a protocol or API, it’s a big warning sign.
So today I forgot to enable the LOGIN authentication mechanism in the SSL connection of my SMTP mail server. The following happens with Windows Live Mail when sending a mail:
- it tries to authenticate but fails silently
- it tries to then send the email directly (like a spammer would do)
This is a good example of over engineering which comes back to bite you hard. Trying to send an email directly (that is, connecting ot the SMTP server of the recipient) should work in theory. In practice, most servers will reject this because most dynamic connections end up in many anti spam lists. The funny thing is that Windows Live Mail is completely silent about this. It doesn’t even show a warning or anything and if I didn’t check my mail logs, I wouldn’t have figured out why a client’s email messages would frequently fail to reach their destination.
But of course the real reason is that email is a mess.