How to Speed Up Uploading Multiple Files via HTTP

A few months ago, I worked on a research task to find out how to improve the speed of multiple HTTP connections from one server (playing a client role) to another server (playing a server role when uploaded files are eventually saved) at the same time. In this specific issue, both servers belong to the same network. That is, we can make any change we want though allocated resource does matter.

After measurement and looking at a few layers, we found out three points to improve this connection:

Implement persisent connections (keep-alive) on the server side, then the client side. If this is not implemented, after uploading a file, the related TCP connection is closed. When the next file is uploaded, a new TCP connection is constructed and this process takes quite a bit of time. It even takes more time than transferring the huge data via our network.
Implement con-currency connections. That is, instead of using a single TCP connection, the client side starts a few connections at the same time, and send data via these connections. We may test and choose the optimal number to optimize the concurrency.

Continue reading →

Java – My First Sip

When reading Head First Design Patterns, I need to run a few examples in Java. Plus that I also want to learn more about Java for two reasons: it’s well-known as a strongly typed programming language, and I have heard quite a lot that it’s a good language to learn OO (object-oriented) deeply.

Some notes:

I use Mac and I got this error when trying to run javac or java: The operation couldn’t be completed. Unable to locate a Java Runtime. That’s because Mac does not come with JDK by default. I needed to download JDK here https://www.oracle.com/java/technologies/javase-jdk16-downloads.html
Here is the GitHub repo containing examples https://github.com/bethrobson/Head-First-Design-Patterns. I tried a few things to run it but got this error: Error: Could not find or load main class headfirst.designpatterns.strategy.MiniDuckSimulator1. Reading this post helped me solve the issue.

Finally, these commands make Java examples of the book work:

$ cd ~/path/to/Head-First-Design-Patterns/src/headfirst/designpatterns/strategy
$ javac *.java
$ cd ~/path/to/Head-First-Design-Patterns/src
$ java headfirst.designpatterns.strategy.MiniDuckSimulator

Quack
Squeak
<< Silence >>
I can't fly
I'm flying with a rocket

PHP: in_array is very slow for a huge array

Definitely not something new but this experience gave me good reason to understand how a function works under the hood.

in_array() goes through all items in a provided array and see if any existing item exists in the array. It’s not an issue if the array has only a few items. However, when it has thousands or even million items, this function is no longer usable!

Instead, a good idea is to flip values to become keys and vice versa. It’s pretty simple if all values are unique then we can just use array_flip() for this purpose and search with new keys (or old values). Good trick indeed! The reason under the hood is that array keys are implemented with Hashtable.

Otherwise, it may be a bit more complicated and we may need to do some other tricks but it can still work!

This conversation suggests using isset(). However, basically it’s still my idea above with a different way telling about that. Even though, it is still a good read!

Test Double

Recently, I have written quite a bit of mock and stub for our REST API services and integration tests. I had some general ideas before, but I have not really got into mock, stub, fake, etc.

And this is a good opportunity for me to get into them seriously. The following is what I’ve read to get into them.

Continue reading →

Book: The Pragmatic Programmer

In terms of purely technical, I would say Code Complete has done better. However, this book is great at the point that it’s really practical and drives software engineers not just focus on technical parts. In addition, you need to understand the project itself, its requirements, how to communicate with other engineers as well as customers.

Still worth to give it a read! I am going to list out what caught me up most.

Continue reading →

Algorithms Specialization Notes

I have finished three first algorithm courses in this specialization on Coursera. My main motivation was to fill up my foundational knowledge of computer science. Generally, I enjoyed all courses and their videos though sometimes I could only capture concepts and ideas behind all proofs. However, I am sure that I understand the motivations and steps to practice all algorithms, and translate them into real code.

Continue reading →

My first experience with E2E: Jest and Puppeteer

I have heard a lot of E2E tests and how important they are to make products, especially the front-end side, more robust. However, I did not have any chance to work with them.

And just in time, when working in Edit Flow, there was a strange issue that E2E tests failed, then suddenly worked, and then failed. I worked on it and learned a few interesting things.

Continue reading →

First Experience with CI: GitHub Actions vs Travis CI vs CircleCI

I did not have any experience with CI until recently. The first task was quite easy when I need to configure the auto-deploy process from GitHub to a site on VIP. It was pretty straightforward as the most complicated task (script for CI) was created already.

The second task was to convert current CI tests from Travis to GitHub Actions (GHA), I have worked on Edit Flow, which is an over-10-year-old plugin. A challenge for me is that tests on Travis all failed.

Continue reading →

WhiteHat Security Trainings

Due to the requirement in my department (VIP/Automattic), I’ve had a chance to work through a few security training courses on whitehatsec.com. This post is more a concise summary for what I’ve got from them.

These are the courses I’ve finished:

OWASP Top Ten for Developers
Building Secure JavaScript Applications
Defensive Enterprise Remediation Series
Integrating Security Throughout the SDLC
Threat Modeling

The first three courses are more practical for developers with exploitation examples and solutions. The last two ones are more about procedure, strategy, and planning.

Continue reading →

PHP 8.0: Non-strict Comparisons for a String and a Number

Non-strict comparisons in PHP are always not easy to work with, and here is the last piece of this common issue:

PHP < 8.0: “one operand is a number and the other one is a numeric string, then the comparison is done numerically”.
PHP >= 8.0: “Non-strict comparisons between numbers and non-numeric strings now work by casting the number to string and comparing the strings.”

Continue reading →