Releases of MakeStaticSite are issued whenever there is some significant change in functionality or a major bug has been fixed. Technically, for a prototype, version numbers start with a 0 and should be regarded as pre-releases.
We summarise here some of the recent key developments, with links to the respective GitHub release pages.
Version 0.30 (current)
Up to: 0.30.13.1
The current version introduces native support for the Wayback Machine, built on top of MakeStaticSite’s established code base. It is designed to support any installation of the Wayback Machine software, though work has focused mainly on the popular deployment on the Internet Archive. The aim is to overcome limitations encountered in the use of third-party tools, as incorporated in version 0.27, combined with the convenience of viewing static output offline.
Since version 0.30.3, MakeStaticSite provides support for crawling sites across multiple timestamps, the depth of capture being determined by the value of the constant, wget_extra_urls_depth. Likewise, associated page elements such as stylesheets and images can be fetched from any timestamp. In version 0.30.6, support was added for date ranges, enabling finer control over the output requested from the Wayback Machine.
Support for WARC, a Web archival standard, was introduced in version 0.30.13. Also, for archival purposes, the mirror output directory can be tailored to indicate precisely the provenance of the Wayback site, particularly the timestamps of both the Wayback capture and the MakeStaticSite run. Various other refinements provide clean output, designed in particular for website recovery, including robots.txt and sitemap.xml files based on the original domain.
It is recommended that when using a public service such as the Internet Archive, to adjust the Wget options to moderate the requests, e.g., by reducing the --limit-rate setting.
There is also more general maintenance including a stricter filter on asset extensions for Wget runs in phase 3 and fixes for issues in the conversion of internal links.
Version 0.29
Up to: 0.29.9
The previous version included some refactoring of code and extended the postprocessing of output from the initial run of Wget (phase 2). It included support for scheme relative URLs, CORS, and assets with long filenames.
In general, the way assets are determined for localisation is more refined: in particular, with more options to restrict the processing, there should be fewer commissions (or false positives) in transforming absolute URLs to relative URLs.
This release laid some foundations to further extend MakeStaticSite’s ability to more thoroughly crawl and create offline replicas, both of more modern websites and sites archived on the Wayback Machine.
Version 0.28
Up to: 0.28.7
This version extended support for authentication and credentials management. When running setup, by default, passwords are no longer stored in the .cfg configuration file, but in a dedicated storage area with restricted file permissions. Furthermore, these credentials can be encrypted with the pass command (to enable this, in constants.sh, set credentials_storage_mode=encrypt).
Version 0.27
Up to: 0.27.8.2
This version added support for crawling the Wayback Machine using the Wayback Machine Downloader (this tool, authored in Ruby, needs to be installed first). As it is incorporated in the phased workflow of MakeStaticSite, the user can benefit from various post-crawl refinements. However, for sites that are more than a few pages, the output will typically be less satisfactory than accessing the original. This may be especially noticeable in browsing due to the design of the Wayback Machine: its crawler usually captures only particular snapshots at a time, which its web server subsequently pieces together, maintaining navigation for the online user. At present, it’s not known if the custom download tools offer functionality analogous to Wget to fetch page requisites, adjust links, etc.
See the Wayback Machine section in Digital Preservation for general discussion, mainly from a user perspective.
There was also improved workflow support for mirroring sites together with assets from extra (or external) domains plus some bug fixes, particularly for running the setup.
Version 0.26
Up to: 0.26.1
This version enabled the setup script to run in unattended mode, simply by running it with the -u flag followed by the URL to be crawled:
./setup.sh -u <url>This command proceeds to automatically invoke makestaticsite.sh, to build a site.
Version 0.25
Up to: 0.25.6,
This version incorporated assets from other domains, typically hosted externally. It also enabled further processing of Wget output for sites where URLs contain paths, moving parent directories into a designated assets folder and updating internal links accordingly. This means that you can bring together under ‘one roof’ the main site, such as a CMS hosted on the primary domain (or a subdirectory), together with the supporting assets, such as those hosted on a CDN. Having done so, they can all be deployed to the same host, CDN, etc. or shared as a zip file.
Version 0.24
Up to: 0.24.1
With a focus on usability, the setup script was enhanced with three run levels, which determine the amount of customisation (i.e. number of questions) available to the user. It ranges from level 0 (where only the URL has to be entered) through to level 2, full customisation with up to about 30 or so options.
Version 0.23
Up to: 0.23.1
The code was refactored in version 0.23, mainly to ensure that the constants.sh file is better able to support potential graphical user interfaces for the setup.
Version 0.22
Up to: 0.22.2
This version 0.22 added support for deployment to
Netlify’s CDN,
This was the first version to be formally placed under Git
version control and stored on GitHub.
Version 0.21
This version focused on web standards compliance for deploying the site live. It improved support for robots, canonical URLs and site maps. In particular, it reconstitutes the canonical URLs as Wget generates them with relative URLs of the form:
<link rel="canonical" href="#">MakeStaticSite replaces these with the respective absolute URL, which for this page is:
<link rel="canonical" href="https://makestaticsite.sh/developers/releases/">