Fast static website creation and deployment
Over the past few years, I’ve been building static websites from starter CSS frameworks such as Bootstrap and Foundation,
iTunes for app developers
The latest version of iTunes doesn’t have app management features. The previous version, iTunes 12.6, allows you to manually upload and download some apps from your iPhone.
Get this custom version of iTunes from https://support.apple.com/en-gb/HT208079
For further reading, look at this article from 9to5mac.com.
What JavaScript libraries is the Internet using?
Have you wondered what JavaScript libraries are most popular? The engineers at Stripe built a website for this, https://libscore.com/. They show the most popular JavaScript libraries as well as the most popular websites.
Nginx configuration to enable CORS for mobile app API
We’ve been working on the Cash Expenses mobile app and building an API to a backend server. Developing this API in our local instance is as trivial as running Python Django’s server on port 8000. For production, we run the Python Django application using gunicorn for the WSGI server, and nginx for listening on the standard web port 80. While the API worked perfectly with the local instance, the API calls were failing when running on production. We traced the problem to our Nginx configuration.
In this case, the mobile apps are calling into the backend service without using Django’s cookie or session authentication; instead, we use Django Rest Knox for token authentication (Django REST Framework’s builtin token authentication should not be used because it is a unencrypted single token).
The Nginx configuration has to be configured to allow all of GET, POST, PUT, PATCH, and DELETE methods from a non-web client. The configuration is inspired by other developers posting their Nginx configurations in Github, and we’ve extended it here:
https://gist.github.com/rhfung/12fced0c159572f5207a2da5b6ecdab1
Upgrading from Webpack 1 to Webpack 2
Webpack is an awesome Javascript transpiler for writing ES6 code that can run on ES5-compatible web browsers. Webpack’s popularity arose with React’s adoption. It’s not limited to React web apps, though: it can transpile any Javascript web application such as Angular 1.x.
I have an existing Angular 1.x web application that I transpile using Webpack 1. Since Webpack 2.2 has entered into release candidates with no further features to add as of December 14, 2016, it’s the right time to upgrade my web application. To upgrade from Webpack 1 to Webpack 2, I followed the the migration notes:
https://webpack.js.org/guides/migrating/
And I referred to the configuration docs for Webpack 2:
https://webpack.js.org/configuration/
The upgrade took less than 2 hours to complete for my project. In addition to upgrading webpack, I had to update a few dependencies (sass-loader, babel-loader) and get some directly from the github repository (extract-text-webpack-plugin) since the Webpack 2.2-compatible release hasn’t launched yet.
Initially I ran into a problem where Webpack didn’t run properly, but removing node_modules directory and running npm install solved that problem.
Overall the consistency of the configuration file is better than in Webpack 1.
Create a PFX File
To create a .pfx file for use in Windows IIS to enable https or for other uses then you need the following files:
- a private key file
- an intermediate certificate from the certificate provider (.crt or .cert) e.g. GoDaddy’s intermediate certificate: gd_bundle-g1-g1.crt
- a certificate
Getting NPM installed inside a Docker container
We’ve been using Docker 1.12 (for Mac) to set up a front-end development pipeline. We found that NPM runs very slowly, if at all. This specific version of Docker has a Docker (virtual) machine behind the scenes, which is installed with the regular Docker installation.
To solve the problem, in our Dockerfile we need to configure NPM settings:
1
2
3
4
5
6
7
8
9
10
11
12
13
|
FROM ubuntu:16.04
RUN apt-get update && \
apt-get install --reinstall -y ca-certificates && \
update-ca-certificates && \
curl -sL https://deb.nodesource.com/setup_4.x | bash
RUN npm config set registry http://registry.npmjs.org/ && \
npm config set strict-ssl false && \
npm config set maxSockets 8 && \
npm install --unsafe-perm --allow-root --ignore-scripts -d
# ...
|
Setting up Python development environment on Mac OS X
Setting up Python on Mac OS X takes a few steps, which will allow you to configure a sane environment that won’t mess up the pre-shipped version of Python.
Remove any old environments
Make sure to remove old installations of Python, if previously installed on your system, except for the one installed by Mac OS X. For example, if we previously installed Python from homebrew, then the uninstall commands are:
1
2
3
|
brew uninstall virtualenv
brew uninstall virtualenvwrapper
brew uninstall python
|
If you installed python via the OS’s python (for example, the one coming from Mac), then:
1 |
sudo pip uninstall virtualenv virtualenvwrapper
|
A side effect of removing previous Python installations is also removing project-specific virtualenv‘s. If you have created requirements.txt files for these environments, it should be straightforward to rebuild them later.
Installing the new environments
Since many versions of Python have been shipped, it’s wise to choose a recent installation of 2.7 and 3.5. We’ll use a environment tool to properly build and install the right version of Python. We will be using pyenv tool. We don’t use homebrew because it doesn’t list all available versions.
1. Install homebrew for Mac if you don’t have it.
1 |
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
|
2. Install pyenv using homebrew:
1
2
3
4
|
brew update
brew install pyenv
brew install pyenv-virtualenv
brew install pyenv-virtualenvwrapper
|
3. Let’s configure Pyenv correctly in your terminal’s ~/.bash_profile (~/.bashrc on Ubuntu). Add the following lines if not already there in the file:
1
2
3
4
|
# Python from pyenv
if which pyenv > /dev/null; then eval "$(pyenv init -)"; fi
if which pyenv-virtualenv-init > /dev/null; then eval "$(pyenv virtualenv-init -)"; fi
pyenv virtualenvwrapper
|
4. After installing the prerequisites, now we’ll use pyenv to install Python. Please choose one of the following steps to follow, depending on which Python you wish to install. Also, if you are using certain packages such as bcrypt, use the alternative install steps.
Regular install steps for Python 2.7.
1
2
3
|
# Python 2
pyenv install 2.7.10
pyenv global 2.7.10
|
Steps for installing Python with bcrypt package (see bug report).
1
2
3
|
# Python 2
PYTHON_CONFIGURE_OPTS="--enable-unicode=ucs2" pyenv install 2.7.10
pyenv global 2.7.10
|
Regular install steps for Python 3.5.
1
2
3
|
# Python 3
pyenv install 3.5.1
pyenv global 3.5.1
|
5. To configure virtual environments for project-specific Python dependencies:
1
2
3
4
5
6
|
mkvirtualenv <your env name>
workon <your env name>
# … Do your work ...
source deactivate
|
Gotcha: We have to use the pyenv version of virtualenv and virtualenvwrapper. The non-pyenv version won’t work: it will stop your terminal from opening (See bug report).
Install Let’s Encrypt on Debian/Ubuntu with Nginx
This post combines several different sources of information on installing Let’s Encrypt on Debian/Ubuntu and configuring SSL on nginx and will show how to install Let’s Encrypt on Ubuntu with nginx already installed. As of writing there is no automated install process provided by Let’s Encrypt for nginx.
For more detailed information see the sources at the end of this post.
Before You Begin
- Make sure the server has at least 2GB of RAM. If not, add or increase your swap size to meet this requirement, otherwise problems may occur during install as reported here and here.
- Update the server software packages:
1 |
sudo apt-get update && sudo apt-get upgrade |
Clone and Install Let’s Encrypt
- If git is not installed then execute the following command:
1 |
sudo apt-get install git |
- Clone the Let’s Encrypt GitHub repository to /opt/letsencrypt:
1 |
sudo git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt |
Generate an SSL Certificate
- Navigate to /opt/letsencrypt:
1 |
cd /opt/letsencrypt |
- Run Let’s Encrypt with the --standalone parameter. For multiple domains add -d example.com:
1 |
sudo -H ./letsencrypt-auto certonly --standalone -d example.com -d www.example.com |
Note:
1
2
3
|
sudo -H
Request that the security policy set the HOME environment variable to the home directory specified by the target
user's password database entry. Depending on the policy, this may be the default behavior.
|
From: Sudo Manual
- Next enter an administrative email address and follow the prompts:
- Agree to the Terms of Service:
- If no problems occur then a similar message will show below:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
IMPORTANT NOTES:
- If you lose your account credentials, you can recover them through
e-mails sent to somebody@example.com.
- Congratulations! Your certificate and chain have been saved at
/etc/letsencrypt/live/example.com/fullchain.pem. Your
cert will expire on 2016-03-31. To obtain a new version of the
certificate in the future, simply run Let's Encrypt again.
- Your account credentials have been saved in your Let's Encrypt
configuration directory at /etc/letsencrypt. You should make a
secure backup of this folder now. This configuration directory will
also contain certificates and private keys obtained by Let's
Encrypt, so making regular backups of this folder is ideal.
- If you like Let's Encrypt, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
|
- The files that will be used in nginx for the certificate are located at /etc/letsencrypt/live/example.com. The two files that are need are fullchain.pem and privkey.pem.
Generate Strong Diffie-Hellman Group
- To increase security, generate a strong Diffie-Hellman group. To generate a 2048-bit group, use this command:
1 |
sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048 |
- This will take a while to generate.
Configure SSL in Nginx
- Edit the nginx configuration that contains your server block. The default one is located at /etc/nginx/sites-available/default, for the purpose this post we will edit this one:
1 |
sudo nano /etc/nginx/sites-available/default |
- Now locate the following lines and comment out or remove them in your server block:
1
2
|
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
|
- The following is an example configuration:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
# http-redirects to https
server {
listen 80 default_server;
server_name secure.example.com;
return 301 https://$host$request_uri;
}
# standalone HTTPS-server
server {
listen 443 ssl;
server_name secure.example.com;
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_dhparam /etc/ssl/certs/dhparam.pem;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;
ssl_stapling on;
ssl_stapling_verify on;
add_header Strict-Transport-Security max-age=15768000;
...
}
|
OCSP Stapling for Let’s Encrypt Certificate with Nginx
- Determine which of the Let’s Encrypt certificates was used to sign your certificate:
1 |
openssl x509 -noout -text -in /etc/letsencrypt/live/example.com/fullchain.pem | grep Issuer: |
- The output will be similar to the following:
1 |
Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3 |
- That last part, Let's Encrypt Authority X3, is the name of the Let’s Encrypt certificate that was used to sign the certificate.
- Download the PEM version of this certificate. All of the Let’s Encrypt intermediate certificates are on the Let’s Encrypt site; click on the “PEM” link for the appropriate certificate to get the file you need. Or, from the command line,
1 |
sudo wget -O /etc/ssl/certs/lets-encrypt-x3-cross-signed.pem "https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem" |
- Replace “x3” with a different certificate name if necessary.
- Edit the nginx configuration from before and add:
1
2
3
|
ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/ssl/certs/lets-encrypt-x3-cross-signed.pem;
|
Apply the Changes
- Test the nginx configuration:
1 |
[sudo] nginx -t |
- Reload nginx to put the changes into effect:
1 |
sudo service nginx reload |
Test the SSL Certificate
- Check with the DigiCert SSL Installation Diagnostic Tool at: https://www.digicert.com/help/
- Check with Qualys SSL Labs Report to see how your server configuration scores at: https://www.ssllabs.com/ssltest/analyze.html (should result in a A+ rating, it will show “OCSP stapling: No” here is the explaination of why)
- By command line:
1 |
openssl.exe s_client -connect [yoursite.com]:443 -status |
- If OCSP stapling is enabled, in the OCSP Response Data section, it should say the following:
1 |
OCSP Response Status: successful (0x0) |
Renewing SSL Certificates
Go to:
1 |
cd /opt/letsencrypt |
Then enter:
1 |
sudo -H ./letsencrypt-auto certonly --standalone --renew-by-default -d example.com -d www.example.com |
It will show a similar message below, certificates have a 90-day lifespan before they expire:
1
2
3
4
5
6
7
8
9
|
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at
/etc/letsencrypt/live/example.com/fullchain.pem. Your
cert will expire on 2016-03-31. To obtain a new version of the
certificate in the future, simply run Let's Encrypt again.
- If you like Let's Encrypt, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
|
Sources:
Software engineering practices to reduce software defects
One of the goals of the software engineering industry is to produce high-quality defect-free software. Software defects get introduced if a programmer makes mistakes in their assumptions. To reduce the possibility of a software defect, programmers should follow best practices such as code reviews, manual testing, automated testing, and canary testing.
Code Reviews
Programmers conduct code reviews to ensure best practices are followed. Code reviews are useful primarily to ensure good coding patterns are followed. Good coding patterns include defensive code to handle invalid input, separation of logic, meaningful variable names, and using proper libraries. Most importantly, code reviews allows a programmer to check assumptions made by other programmers. Contrary to popular belief, code reviews do not reliably find software defects.
Testing
Most software defects are found by testing. Testing runs the software in a controlled environment. Different inputs are provided to the software and checked against expected output. The software should also be checked against invalid input to ensure it gracefully fails. Testing these input and outputs can be performed by manual or automated testing.
Manual Testing
Manual testing only verifies that the software works at a specific instance in time; after the next code change is made, the tests will have to be performed again. Given that software is changed several times a month, if not every day, significant time and effort is required for manual testing.
Automated Testing
Automated testing ensures that the software continues to work after any code change. Automated testing can be performed by unit tests or integration tests. Unit tests check the smallest chunks of code and integrations tests check entire end-to-end systems. Although there is a significant time to write automated tests initially, it saves time in the long run by making new feature development more stable without breaking existing code.
Canary Testing
A specific type of test is a canary test, which has a higher degree of confidence than automated testing. The software is tested against a copy of real-world inputs. An engineer’s set of test inputs might have missed some assumptions of real-world inputs, which will be caught in this type of testing. For example, canary testing can test algorithm speed and memory usage. The limitations to canary testing are computational resources and company approval to access customer data.