Dev Insights

This is a log of developer insights and progress.

001: What is Electron?

Online references do a poor job of differentiating Electron from Electron tooling and utilities. Many of the affiliated "userland" related packages (e.g. electron-forge vs electron-builder vs various boilerplates) overlap in purpose and don't describe how they work in a systematic way. Complicating the picture is understanding how Webpack works (dev tool) with node (dev tool) and npm (dev package tool), and keeping that separate from Electron and Electron Userland.

The following is my hard-won breakdown of what I think Electron is, how it works with Webpack, and how both of them work with Node and npm.

The Essence of Electron

The Electron npm package has two parts: a runtime binary that is NodeJS combined with Chromium, and a NodeJS library that provides helpers to integrate with the native operating system.
Electron allows you to write web applications using HTML/CSS/JAVASCRIPT that run in "renderer" (browser window) processes that are like tabs in Chrome. These web applications, running in an Electron renderer process, can also access any NodeJS feature! This makes the system very powerful and self-contained.
To write Electron code, you require the 'electron' library in your NodeJS program, then access its various helper objects to spawn Browser Windows and access operating system features. In this way, you can make your web application work like a desktop app.
To distribute an Electron App, you bundle the Electron runtime binary with your files. There is a separate tool called electron-builder that will automate this process for you.
Likewise, there are many other tools that purport to simplify Electron development for you. However, these tools are NOT part of Electron and do not help you develop an understanding of essential Electron development concepts. If anything they obfuscate what Electron actually does using out-of-date tools and documentation.

A simple Electron app consists of two files: a NodeJS program (e.g. main.js) that uses the electron library to open an HTML file (e.g. index.html) into a Chromium BrowserWindow. The HTML file can then load its own assets (javascript, css, etc) like any web page running in the Chrome browser. The additional superpower that Electron-hosted web pages gain is access to the NodeJS environment in client-side Javascript, meaning your web page can do all kinds of interesting things that are usually forbidden by the browser for security reasons.

Note that your local web pages are accessed as FILES, but you can also access assets via HTTP/HTTPS as well. However, since Electron provides full access to your NodeJS environment, be careful about what you load.

Webpack and Electron

More advanced web app development uses a build tool like Webpack, which is designed to be used from the command line.

Webpack is a tool that enables modular javascript WebApp development.
Webpack automagically rewrites individual .JS files into into a compact "bundle" file that is ready for use in a web server.
Webpack only makes bundles, but it can also do transformation as it bundles (e.g. converting JSX to JS through Babel) through its extensive "loaders" and "plugins" ecosystem.
Webpack itself does not handle development serving or livereloading, but webpack-dev-server is a utility that does this.
Webpack is often invoked through NPM scripts defined in package.json, which acts as the primary task runner for development compilation and debugging tasks.

Important Webpack Concepts:

The best concise overview of how Webpack works is Webpack Concepts. Don't waste your time searching for other tutorials on Webpack until you have read this.
Concept: Webpack is a NodeJS-based tool. The Webpack configuration files are written in Javascript and executed by Node, exporting a configuration object.
Concept: The Webpack configuration file is an executable Javascript file that exports an object. It has access to NodeJS and NPM packages, so this is where you can "script" Webpack itself.
Concept: Webpack basic operation starts with a defined "entry point" file, scanning it for other included javascript, css, or other assets. Ultimately, these files are "bundled" together in the dist folder by default and are ready to be served to a web browser.
Concept: Webpack "tests" tell it how to process each kind of file using a "loader". For example, .js files can be handled by babel-loader before it's bundled. Here's a list of available loaders.
Concept: Webpack has "plugins", which can hook into different stages of the Webpack lifecycle. See the list of available plugins, or for deeper understanding see the API Reference for the various Compiler and Compilation hooks available.
Concept: Webpack configuration files are often designed to be "merged" together, so you need to understand the webpack-merge and Webpack's default values to make any sense of it. You also need to understand the concepts above.

The Untamed Electron Userland

I mentioned earlier that there are various official "userland" tools. These are packages that area not part of the actual Electron library, but use it or extend it. They are not good for developing a working understanding of how the tooling all fits together.

The starting point I used was Boilerplates and CLIs on the official Electron website. Boilerplates are command-line development environments that may or may not use one of the Electron Userland packages. Many of them are using out-of-date versions of Electron, out-of-date web tools, and don't describe any of the theory. If you are not already experienced with the way of NodeJS development, Webpack, and the history of web application development practices, it will be difficult to follow because you have to infer the meaning of terms like "production" and "development", and what specifically that means for a Webpack+Electron application. You'll have to know the webapp build process and its typical stages of the transformation pipeline too.

What was helpful for me in understanding the overall shape of Electron development was installing 6 different repos that purportedly all did the same thing. These packages were:

electron-forge: A binary package that's supposed to be an easy way to scaffold a new Electron project, but it is currently between versions. You use it from the command line. The version 6 beta is the current branch, but is difficult to install and is not a good candidate for our project because of its broken state.
electron-builder: A binary package similar to electron-forge, but more of an eye toward packaging, deploying, and updating an Electron app on multiple desktop and server platforms. It is worthwhile to see its feature set that goes beyond basic Electron to get an idea of what you could do, but it is for the advanced user. It uses several other Electron Userland tools like electron-compile (a packager, I believe).
electron-webpack-quick-start: This is a boilerplate, not a tool, meaning it's essentially a package.json file with some included "hello world" examples that you can immediately run as an Electron App. Behind-the-scenes it just uses electron-builder.
electron-quickstart: Another variation on boilerplate.
electron-react-boilerplate: Another variation on boilerplate, but with REACT added. This is like electron-quickstart with additional Webpack configuration to include React.
electron-react-webpack-boilerplate: Another variation on boilerplate, but with Webpack and REACT added. This really does the same thing as electron-react-boilerplate, but just has a similar name and a different package.json script setup.

None of these were very helpful in designing our own dev environment because they are all "use and don't understand" approaches to software development. Going to first principles required studying Webpack configuration to see what the tools actually did. The process I followed to gain this understanding looks much like this:

Look at the package.json for each of the boilerplates/tools to see the scripts and dependencies/devDependencies contents.
Look at the referenced Webpack configuration in the npm run scripts. Reading Webpack Concepts was an important starting point.
Concept: what a bare bones Electron App would look like.
Concept: how the order-of-execution in the key source code files of the Electron App.
Concept: what the development pipeline tasks are to build and debug an Electron App with the live-reload and debugging features we want.
Concept: how the MEME WebApp, which is based on PLAE and NetCreate, differs from the basic Electron App.
Concept: designate places in the pipeline and order-of-execution that insert our WebApp into ElectronApp with all desired features.

002: Proposed Boilerplate

After incorporating Webpack into our development workflow, we have the following tasks to implement through the npm run script interface:

REVISED 11/1/2018

TARGET: webapp accessible via browser, served from a local webserver. The local webserver can be run in a development mode, or deployed as a packaged binary.

npm run dev builds webapp from src/web-app through webpack-dev-server (WDS); view at localhost:3000. Livereload works for both html and javascript changes. Also writes files to dist/, which WDS does not do by default.
npm run electron builds electronapp from src/electron-app and runs it through the Electron binary.
npm run package creates an installable stand-alone package.
npm run clean removes the 'dist' directory
npm run clean:runtime removes the 'dist' and 'runtime' directories that hold saved data.
npm run install runs the necessary npm ci command on first-time use
npm run install:clean removes the node_modules folder then runs DEV:INSTALL

003: Electron Integration

The webpack-dev-server works well for webapp development, but for Electron to host files with livereload there are a few options:

use the API version of WDS, with additional webpack-dev-middleware options from within electron-main.js
spawn the CLI version of WDS and its own config file from electron-main.js
write a custom runtime server that uses express, chokidar and webpack (watch mode) for both webapp and electronapp sources. The server has to inject a client script to handle reloads on the client side of things.