- Technology Stack
- Why?
- What?
- tl;dr
As a team of people
using technology
to make digital products,
it's essential to be unambiguous
about the stack/tools we use,
so that everyone is clear
what we all need to master.
If anything is unclear or you have any questions please ask.
This document + diagrams describe
the full "PETAL" Technology Stack
we use for dwyl products/projects.
Each element in our stack was carefully selected based
on its individual merits.
When assembled into a seamless "machine",
the stack is unrivaled for developer productivity
and world-class quality!
"PETAL" is an acronym1 for the following elements:
Phoenix is a Web Application Framework
that does not compromise
on speed, reliability or maintainability!
Phoenix is the "successor"
to the incredibly popular "Ruby-on-Rails" framework.
Built from scratch by highly experienced engineers
who worked on/with Rails. It solves
all of the speed/socket/scaling/concurrency, issues
people felt when building/using Rails apps.
The list of benefits Phoenix has over
(virtually every) other Web Frameworks is extensive.
Please see:
dwyl/learn-phoenix-framework#our-top-10-reasons-why-phoenix
Phoenix tops the list of "Most Loved" Frameworks
on the 2022 StackOverflow Community Survey ❤️
Elixir is the functional programming language
used by the Phoenix framework.
Elixir is a beautiful language
written from scratch to be
friendly, concise and efficient.
Yes, Elixir not as
"mainstream"
as JavaScript, Java, C# or PHP,
but the adoption is growing rapidly and most importantly
many experienced developers are gravitating towards and
describing it as their "most wanted"
Also a language's popularity has more
to do with the intellectual inertia people/companies have because
they allow existing (legacy) codebases to dictate future development;
i.e.
sunk cost bias.
see: dwyl/learn-elixir#key-advantages
Elixir is the 2nd
"Most Loved" programming language:
This is a good measure of how much people enjoy working in the language. And as we all know people who enjoy their work are better at doing it!
Tailwind is the most sane way
of creating a beautiful web app UI
that can easily be extended by a team of people
without fear of one person's change "breaking" another feature.
Unlike "traditional" CSS which - as it's name implies - encourages
"cascading" of styles, Tailwind
makes the style of each component specific
and local to that component.
see:
dwyl/learn-tailwind
Alpine.js is a lightweight library for enhancing interactions
in a web application. It's declarative, responsive and easy to learn.
Alphine.js plays well with LiveView for progressive enhancements.
see:
dwyl/learn-alpine.js
LiveView is a radically simplified way
of building realtime web apps with significantly less code.
We have crafted a "Complete Beginner's Guide" for each element in the stack, so that we:
- Document our collective learning
whilewe are building projects.
(because as humans we forget fast unless we capture it immediately!) - Share our knowledge with other people so we can
- Help to train (potential) new team members as quickly/effectively as possible.
- Collectively iterate on our knowledge and "level-up" as a team!
- "Onboard" the client team (who may want/need) to support/maintain the codebase/project if/when we seamlessly "hand over".
- Inform the wider community of both technical and non-technical people ("stake holders") who are generally interested in understanding the project.
- Enlighten other teams/organisations/agencies/etc. we aren't in direct contact with that there is a "more fun" way of building software!
- Make everyone's life easier/better by having a "launch pad" for rapid learning!
We have written several beginner tutorials that span our technology stack and tools we actively use in development.
Here are a few of our learning repositories
pertaining to Phoenix and Phoenix Liveview.
- Learn
Elixir: dwyl/learn-elixir - Learn
Phoenix: dwyl/learn-phoenix-framework - Counter (Liveview): dwyl/phoenix-liveview-counter-tutorial
- Todo List (Liveview): dwyl/phoenix-liveview-todo-list-tutorial
- Stopwatch (Liveview): dwyl/phoenix-liveview-stopwatch
- Chat: dwyl/phoenix-chat-example
- Chat (Liveview): dwyl/phoenix-liveview-chat-example
- Realtime cursor tracking (Liveview): dwyl/phoenix-liveview-realtime-cursor-tracking-tutorial
PapertrailandPhoenix: dwyl/phoenix-papertrail-demoFlutterandPhoenix: dwyl/flutter-phoenix-channels-demo
We have a couple of "internal" (but Open Source) projects
that use Phoenix
and serve as a good showcase for the stack:
- Labels: dwyl/labels
- Hits: dwyl/hits
- Useful: dwyl/useful - utility library.
- Content: dwyl/content - content negotiation.
In this section you will find learning repositories
where you can learn Flutter
and how to use it with other technologies.
- Learn
Flutter: dwyl/learn-flutter - Learn
Dart: dwyl/learn-dart SupabaseandFlutter: dwyl/supabase-flutter-demo- Counter: dwyl/flutter-counter-example
- Stopwatch: dwyl/flutter-stopwatch-tutorial
- Todo list: dwyl/flutter-todo-list-tutorial
Bloc: dwyl/flutter-bloc-tutorial
In this section,
we will list a few repos that explain
concepts and tools that we are actively using
while developing our app.
- Payment processing: dwyl/learn-payment-processing
- API design: dwyl/learn-api-design
- Test-driven development dwyl/learn-tdd
InsomniaAPI client: dwyl/learn-insomniaGithub Pagesdeployment: dwyl/learn-github-pages
We have built a fully working MVP version of our App! Check it out at dwyl/mvp!
The reason we do not specify our Database
in the "PETAL" Acronym is
that Phoenix allows us
to use any Relational Database.
By abstracting the data layer using "Ecto" the application is "decoupled"
from the database.
This means that if a client asks us to deploy to MySQL or
Microsoft SQL Server
(e.g. because they already have in-house capability
for maintaining one of these databases)
we can easily accommodate that
without re-writing any of the Phoenix app!
Our "standard" (preference) @dwyl is for Postgres.
see:
dwyl/learn-postgresql
Postgres is the most "mature" Open Source Relational Database.
It's 100% Free (including all "advanced" features)
and has been deployed and battle-tested in every environment
from AWS to "Bare Metal" and Google Cloud to Microsoft Azure!
Many well-known/successful apps rely on
Postgresas theirmaindatabase.
NOT that you should adopt a particular technology based on whoelseis using it,
but it's good to know that plenty of teams are getting excellent results withPostgres!
We have used most of the "popular" Relational Databases.
e.g: MySQL, Microsoft SQL Server,
Oracle and Aurora, etc;
all
RDBMS
have their pros/cons.
The reason we like/use Postgres
is because the community is superb.
There is a great "bank" of answered questions on
StackOverflow
and new questions get answered fast.
A "traditional"
LAMP stack
includes the Linux Operating System
in the name.
The "PETAL" stack runs
on any (desktop/server) Operating System
and can be deployed to any "cloud" infrastructure provider.
While we have a strong preference
for Unix (e.g. FreeBSD) or
Linux (e.g. Ubuntu or CentOS) we know that
both Phoenix and Postgres run
on almost any environment including
Microsoft Windows Desktop & Server.
We are using GitHub actions
for Continuous Integration /
Continuous Deployment.
For an example of this,
including automatic deployment to Fly.io
see:
.github/workflows/ci.yml
We make a point of deploying our work as soon as there is something worth showing to the target audience of "end users" so that we can get feedback as early as possible.
Lately we have been using Fly.io for deploying our Apps. The experience is superb. ❤️
The Phoenix Application Server is hosted on (a minimum of)
Two Linux Servers.
(often many more which send messages
one another to distribute load as a cluster).
The "cluster" is managed by Erlang's "Supervisor".
The Erlang Supervisor
is the "Gold Standard" in infrastructure management,
having been used by Telecoms companies
for over 20 years in production
with some Telcos reporting 99.9999999%
("nine nines") of "up-time".
It's far more likely that the infrastructure provider (e.g. AWS/Azure) will have a fault in their network/datacenter than an Erlang server "crashing".
All communication is over secure/encrypted channel
(by default at all times)
to protect the data/privacy
of people using the applications we make.
We recommend using the "Let's Encrypt" service for SSL Certificates
it's 100% Free (and provided by a Non-Profit foundation)
to help you get started, we wrote a
step-by-step setup guide for apps deployed to Heroku:
SSL-certificate-step-by-step-setup-instructions.md
There is no shortage of options available for
Technology Stack!
See: https://www.google.com/search?q=technology+stack&tbm=isch
So, how did we arrive at the conclusion that PETAL
was "the one" for us...?
We already had a really good
Node.js Stack
which worked well for us and our clients. so . . .
Our reasoning for
considering an alternative approach/stack for building web apps
was fueled by our
curiousity
and
"shoshin".
"The important thing is not to stop questioning.
Curiosity has its own reason for existence."
~ Albert Einstein
In November 2016 we (once again)
questioned our assumptions,
re-examined and
surveyed
the "landscape" of "emerging trends" in web app development.
We were pleasantly surprised delighted to see the amazing progress
made by the people in the Elixir / Phoenix community!
Please see:
dwyl/learn-phoenix-framework#questions
One of the most "difficult decisions" you will make in your "career" is which technologies and tools you will use to deliver the desired solution/benefit to the "end users".
Most people have the Tech/Tools decision made for them
by the company/organisation/boss they work for
(e.g: Java -> Spring,
Ruby -> Rails
or PHP -> WordPress or Laravel, etc.)
This is because most companies
already have an existing app in "production",
which you have been hired to extend.
Occasionally you will get the chance
to build an app from "scratch"
however most of the time someone else (the "Architect")
will make the decision for what "stack" to use on your behalf,
so you will still be stuck with someone else's choices.
If you are incredibly lucky the "Architect(s)"
will have done their homework: surveyed the latest industry
trends and investigated the "new and promising" technologies
e.g: Stack Overflow
"Most Wanted" list.
-
Go with what you (already) know, use existing stack with a minor variation because it's "easy to deploy" with the existing infrastructure and will not get questioned by the "Executives", DevOps team or "Compliance" department. This is the easy choice and nobody ever got "fired" for sticking with what they know "works".
-
Buy the whizz-bang all-in-one solution sold to them by the "Consultant" from "Big Vendor XYZ" (outsource the thinking to a sales person who last wrote code in the 90's ... seems like a great idea ... NOT!)
-
Be "Bold" and try "Popular Framework XYZ" and hire an external team to build the new magic app. Then attempt to "up-skill" the internal team to maintain the code written by the consultants.
None of these choices is optimal, all have different levels of risk/reward. The "hardest" choice to make is the one where you try something totally different. The reality is that very few people have the time/resources/mindset/inclination to take a step back and open their minds to the idea that there might be a "better tool" for the job than the one they are currently using.
Imagine Want to Make Yourself Some Toast.
The "user story" for this would be:
As apeckish person
I wanta slice of tasty toast
So thatI can have some crunch for my brunch!
( let's assume that you bought a loaf of bread from a baker
to reduce the options for solutions for simplicity
i.e. not baking from scratch! )
The "traditional" way to "solve"
the challenge of making toast:
- Cut bread with bread knife
- Put sliced bread in toaster
- Turn on toaster for pre-determined amount of time
- Wait patiently for toaster to make toast
But ... what if instead the "old" way we just described,
someone invented a way
to toast the bread while slicing it...?!
Simply by using the "New Tool" for the job -
in this case the
"Toast Knife" -
you can
simplify the process to a single step!
This is the power of being open to considering "New" Tools/Technologies!
Get the same result in fewer than half the "steps"!
The short-term cost of learning a new stack (programming language or framework) is time, We contend that the 1 week learning time (depending on the focus of learners) will pay for itself within 1 month (often sooner if the team is large/distributed because the structure offered by Phoenix means everyone working on the project has a disciplined approach to adding new features)
- https://hbr.org/2012/08/thinking-long-term-in-a-short
- https://hbr.org/2011/03/capitalism-for-the-long-term
In 1996 Nokia introduced the
"Communicator"
and was a incredible revolutionary innovation!
Internet, Email and Fax in your Pocket!!
Nokia continued to dominate the mobile phone industry/market for the next
decade producing the best-selling
5110
and
3310
some of us are old enough to remember!
But by being "ahead" Nokia was unable to see the "contender"
coming "out of nowhere" to challenge their position.
In 2006 nobody was making/buying "smart" mobile phones
with glass touch screens that ran "apps" ...
in
January 2007 Steve Jobs introduced the iPhone
and literally changed the industry!
The dominant/incumbent mobile phone maker Nokia had
49% market share in 2007
mocked Apple's lack of features, poor battery life and high price.
By 2013 Nokia had 3% Market Share (for new device sales) and was sold off "for parts" to Microsoft
while Apple was the most valuable company
in history!
Many people still buy "feature phones" (the polite name for a device that does not have any "smart" functionality!)
but few people can convincingly argue that the reason they don't have a smart phone is because they don't want one;
over 90% 16-24 year olds own a smart phone and that number is expected to reach 100% by 2025 ...
ask someone in their 20's if they would go "back" to using a non-smart phone and see what they say ... "No Way!!"
they laugh uncomfortably and admit that: "My Smartphone is my Life!"
We feel exactly the same about "old technology". Sure the "old way still works", but if you can inexpensively switch
to something demonstrably better in every aspect, why would you stick with the "feature phone" of web frameworks...?
It's like taking the Bus to work when there's a perfectly good teleporter right next to the bus stop!! Madness.
We are not suggesting that everyone is going to suddenly flock to the "PETAL" stack the way people adopted smart phones. This is merely an illustration that when a technology has a specific/measurable advantage to it's users the adoption can be fast.
In the case of programming languages or application frameworks, moving one framework to another is a much more difficult decision.
But one thing is for sure we are going to use the "smart phone" even if other people insist on using the "brick".
If you are new to web development,
please focus on UX
and forget about "scale"!
Unless you work somewhere that already has "millions of users" and
your team cannot consider anything that does not support a million concurrent connections...!
But let's face it, most people have imaginary scaling issues not real ones.
discussing "scalability"beforeyou have 10,000 paying customers is a waste of time!!
Stop worrying about "scalability"
and instead focus on building something useful
focus on User Experience not ("backend") scalability!
The good news is that
Phoenix "scales" really well!
see:
phoenixframework.org/blog/the-road-to-2-million-websocket-connections
Forget about "scaling" until you have made
something people want
and are paying for!
Then use the pile of cash you got from your product
to hire "engineers" to make it available to more people!!
We still think that
"Full Stack JavaScript"
is a compelling proposition
especially for people who are just starting out!
It allows you to write one programming language
on both the client and server; we get it!
However we have learned from years of experience
that it requires a lot more code and maintenance
than PETAL for an inferior result
in terms of UX/performance and developer productivity.
If we were to consider an alternative to SQL, we
would use RethinkDB:
https://rethinkdb.com
But we are relieved that the Phoenix team
is focussed on PostgreSQL because that eliminates
the "ambiguity" or "discussion" of "which database" to use!
Postgres is a fantastic "general purpose"
store that has a rich ("structured") query language
that lets you JOIN data!!
Also, now that Citus DB is Open Source
we know that Postgres
can easily handle billions of writes per day!
“If it takes an hour to figure out what’s going on, well,
that’s an hour that wasn’t spent doing something else more useful and interesting."
~ Rachel Kroll
Please read:
https://www.radicalsimpli.city
In the site/manifesto
Stephan
makes the case that Apps in 2021
have gotten far too complex:
He advocates for a return to basics:
No need for microservices message queues or other
complex tech that is only relevant to 0.1% of mega scale companies:
We agree.
If by some luck our product reaches the point where
we need mega scale
(millions of people creating billions of items)
we know that our chosen stack will scale just fine.
Before then the complexity will only slow us down and reduce the chances of success.
We have written about our choice of programming language extensively in: learn-elixir/issues/102.
Our use of Elixir is for a very specific reason:
we are building fault-tolerant realtime systems.
For the type of App we are building,
Erlang/OTP is the undisputed king
on the server side.
We could use almost any other language/framework,
but it would be a lot more work for an inferior result.
If we need to build a specific feature requested by a person using our product, then we will 100% consider a technology that enables us to deliver it.
The way to propose a specific tech/tool is simple: open an issue describe how the tech/tool will help us build a specific feature that has been requested by a person using our product.
Proactively create a new repo
in the dwyl org
to capture your own learning
of the tech/tool you are proposing.
e.g:
dwyl?q=learn
Once you have invested the time
to learn the tech/tool beyond "hello world"
and are confident that it will help us
achieve a specific end-goal,
then please make the case for it.
