SubSonic

We have a Session process that is being Supervised, which is the hard part. Now let's hook up a persistent data source.

When you build applications in the Erlang world you create discrete processes that interact. In theory this is pretty straightforward - until you actually try to do it. Microservices fans out there know the value (and the pain) of managing a fleet of ... [More] services; there are benefits to it, definitely, and also problems. Consider this quote from Benjamin Wootton, describing what he's going through while building a system based on microservices: I am currently involved in architecting a system based around Microservices, and whilst the individual services are very simple, a lot of complexity exists at a higher level level in terms of managing these services and orchestrating business processes throughout them... Microservices [is] one of these ideas that are nice in practice, but all manner of complexity comes out when it meets reality... So very true. I like how Jessica Kerr puts it: Erlang has been providing [the connection between services] for literally 25 years. As we get more and more sophisticated microservice implementations, each one grows their own crappy version of Erlang Erlang has been doing this kind of thing for 25 years. The community has had time to formalize a number of ideas into concepts, which have made their way into a platform: OTP. That's exactly what we'll be using, right now. The First Task: A Shopping Cart It's a fine place to start, why not. The first thing to do is embrace that I'm building a process here, not a set of objects (Cart and CartItem e.g.). If you think about Shopping as a process that your customer engages in - well I'd say we have our first Application: cd apps mix new shopping --module Redfour.Shopping * creating README.md * creating .gitignore * creating mix.exs * creating config * creating config/config.exs * creating lib * creating lib/shopping.ex * creating test * creating test/test_helper.exs * creating test/shopping_test.exs Your Mix project was created successfully. You can use "mix" to compile it, test it, and more: cd shopping mix test Run "mix help" for more commands. We've just created an Application, which is a discrete OTP construct: When you have written code implementing some specific functionality you might want to make the code into an application, that is, a component that can be started and stopped as a unit, and which can also be reused in other systems. You can think of Applications as "components" or "beans" or "dlls". This fits perfectly well with what we want - our Shopping application will implement a number of its own child services, and it will be used by our web app as well (which is another Application). Task Two: The Session We've setup the structure of our Shopping process - but how will it be carried out? As with Real Life, Shopping is just a concept for customers walking around your store, picking things off the shelves. Each one of them is Shopping, if you will. If I were to break the process down, it would look like this: A Customer enters the store, starting a shopping session as they browse our catalog and various promotions They put stuff in their cart, remove stuff, and decide whether they are buying something They take their selections to checkout and buy them (hopefully) We record the sale and all the data that went into it (because we tracked them through the store) for reporting purposes Functional programming is all about moving/transforming data through a one or more processes. Looking at this list I would say that a Session moves through a Shopping process. Let's define that! The first thing to do is to create an application for Shopping: cd apps mix new shopping --module Redfour.Shopping The next thing is to create a directory and code file: cd apps/shopping mkdir lib/shopping touch lib/shopping/session.ex Now let's pop some code in: defmodule Redfour.Shopping.Session do defstruct [ domain: nil, id: nil, key: nil, landing: "/", ip: "127.0.0.1", member_id: nil, items: [], logs: [], discounts: [] ] end This struct defines our Session explicitly. The domain of the store (in case we have more than one, which is possible), an id from our database, a key that uniquely identifies it in a business sense, and lists to hold items, logs and any discounts that they use. To have an effective store we need to track everything - and this is a great first start. Task Three: The GenServer Any code that you write in Elixir runs in a singular process. Even if you try to write some Elixir and quickly run it with iex (the Elixir REPL) - it's still given a process. At the most basic level a process can be this: iex(1)> spawn(fn() -> IO.puts "Hello World" end) Hello World #PID<0.62.0> The spawn command tells Elixir to spin off some code and run it separately. The fn() -> IO.puts "Hello World" end bits are just an anonymous function declared inline - that part's not important - what is important is how easy it is to spawn things that run in their own process and just do stuff. Note: These are VM processes, completely contained within the Erlang VM. Not OS processes. We would have a massive hill to climb if we decided to build our Session using nothing but spawn. We would, as Jessica put it above: build our own crappy version of Erlang. Let's jump ahead and just use OTP: defmodule Redfour.Shopping.Session do use GenServer def start_link(%{key: key} = args) do GenServer.start_link(__MODULE__,args, name: key) end defstruct [ domain: nil, id: nil, key: nil, landing: "/", ip: "127.0.0.1", member_id: nil, items: [], logs: [], discounts: [] ] end By implementing use GenServer here, we've made our Session into a formalized OTP Server that can do a number of groovy things - the most important of which is that it can now be supervised. More on that in a minute. No matter how much I write and wave my arms - understanding GenServer takes some actual doing. It honestly isn't that complicated - but the first few times you encounter this little beasty ... well you're on Google non-stop (at least I was). There's a learning curve here but, hopefully, we can learn as we go. We have our struct defined and have made our Session into a formalized GenServer process. I need to make sure this thing can maintain state, which might sound a little odd given that we're using a functional language; the whole idea of which is to remove the notion of mutation and, basically, the changing of state. That's what GenServers do: they give you a way to attach state to a process. We do that by seeding it with some data when it starts, and passing that data back after each call. With start_link we seeded the data with a map: def start_link(%{key: key} = args) do GenServer.start_link(__MODULE__,args, name: key) end This is telling OTP to create a process for us in the VM with the initial data of args (which is a Map) and a name corresponding to some key. Here's how I would use this: iex(1)> {:ok, pid} = Redfour.Shopping.Session.start_link %{key: :my_session} {:ok, #PID<0.93.0>} iex(2)> Redfour.Shopping.Session.do_something :my_session, some_arguments We now have a process running in memory that I can refer to anywhere, as long as I know the name :my_session. This process will uniquely describe the notion of someone browsing through our store, looking at things, picking them out, etc. You might be wondering what happens when another customer comes and I need to start another Session? Redfour.Shopping.Session.start_link %{key: :new_customer} Now we have two sessions running. How do I know? Let's have a look: iex(3)> :observer.start You should see an application pop up with all kinds of interesting tabs on it. This is the Erlang observer - it tells you what's happening with the VM and all the processes running inside. If we look for ours: Which is nice and all, but not ideal. Just like any thread or process in other languages, if something goes wrong this process will exit and die - along with all of it's state (items in the cart and so on). This is bad. Speaking of "bad" - you might be wondering about scalability here. How can we expect our app to scale if we have all of these processes bouncing around in memory? This is one of the (many) great parts about the Erlang VM: it's incredibly efficient. Each process has its own heap - which basically means that there's no shared memory between processes. When one dies, everything goes with it. When a parent process dies, all of its children die too. Of course we don't want these processes living forever - so we can actually set a timer on them if we want; but I'd rather handle that explicitly (kiling them) - which I'll do in a later post. Let's flip over to the Applications tab: This tells you what OTP Applications are currently running - and so far it's just one: ours. Process 0.82.0 is our start_link and 0.83.0 is init (which I'll talk about more next time). We then have an Elixir.Logger.Supervisor process which, in turn, has spawned a number of child processes that it needs. Our little processes are nowhere to be found in this tree. We need to change that. Task Four: Supervision In Elixir (and Erlang), you don't write code to recover from errors: You let things die. You do this because you can, and it's a natural thing to do if you want a system that stays up and doesn't crash. This might sound completely bonkers at first - but hold tight hopefully you'll see what I mean. What we need for our Application is a Supervision Tree. In OTP, a Supervisor is a process that watches other processes and, if they die, it restarts them. Pretty simple to explain, a bit harder to correctly put together. When I created our Application I made it rather bare bones with no notion of a Supervisor. If I would have added the --sup flag it would have created a supervisor for us - but that's OK we can just do it ourselves. The first thing to do is to change our apps/shopping/lib/shopping.ex main file: defmodule Redfour.Shopping do use Application import Supervisor.Spec, warn: false def start(_type, _args) do #supervision goes here end end Next, we need to formalize this as the startup module by telling Mix. Open up apps/shopping/mix.exs and edit the application callback to define the start module: def application do [ applications: [:logger], mod: {Redfour.Shopping, []} #add this ] end This callback is telling Mix what additional applications to start, aside from our own. You can see the :logger here - which is what we saw in the :observer above. In addition to these applications, we want Mix to run our Redfour.Shopping module, passing in an empty argument list. Now, let's flip back over to shopping.ex and add our Supervisor code. I'll do this in two separate functions: defmodule Redfour.Shopping do use Application import Supervisor.Spec, warn: false #the entry point to start our app def start(_type, _args) do #supervision goes here start_session_supervisor end #define our parent supervisor def start_session_supervisor do #spec the session supervisor session_worker = worker(Redfour.Shopping.Session, []) Supervisor.start_link([session_worker], strategy: :simple_one_for_one, name: Redfour.SessionSupervisor) end def start_session(key: key) when is_binary(key), do: raise "Please use an atom key" def start_session(key: key) when is_atom(key) do res = Supervisor.start_child(Redfour.SessionSupervisor, [%{key: key}]) end end This took me a while to figure out - hours to be exact. It's a little impenetrable and rather hard to find resources online to properly explain it all - but I'll do my best. The start method is called by Mix when the app starts up. This, in turn calls start_session_supervisor which defines a worker - a thing which will be supervised. This is just a spec, it's not actually starting up the process. I'm using a :simple_one_for_one strategy which means, basically, "if the worker dies, restart it". Finally I'm giving it the name Redfour.ShoppingSupervisor. Now for the good stuff. I'm declaring a method called start_session that will spawn a worker within the purview of a SessionSupervisor. Every time I call this, the SessionSupervisor will call Redfour.Shopping.Session.start_link %{key: key} - creating our GenServer on the VM. Let's give it a whirl. Navigate into the apps/shopping directory and start things up with iex, making sure to tell it to load up the mix.exs file using -S mix: iex -S mix #... iex(1)> Redfour.Shopping.start_session key: :my_key {:ok, #PID<0.96.0>} Yes! Let's have a look at the observer again - run :observer.start within iex one more time. Don't close iex! You should see this in the Applications tab: Here, 91 and 92 are our application starting up. Then we have our Supervisor, which is a child process of our application and finally our Session, identified by our key my_key. Let's add another session to be sure we've done things correctly. Keeping iex open, start another Session: iex(3)> Redfour.Shopping.start_session key: :another_key {:ok, #PID<0.676.0>} Radical! Now let's prove our theory. Keeping iex open still, let's kill a process off and make sure it gets restarted as we expect: iex(4)> pid = Process.whereis :my_key #PID<0.96.0> iex(5)> Process.exit pid, :kill true iex(6)> Process.whereis :my_key #PID<0.683.0> I can grab a running process by name using Process.whereis and get its pid - here you see it's 0.96.0. I then use Process.exit passing in the pid and the directive to kill it off, which indeed happens. If I ask for the process again - I can see it's running once more with a new pid of 0.683.0! You can confirm that it's been restarted looking back at the :observer. Next Time: Working With Data A Session that's built to stay alive with a unique key assigned. Let your brain wander on that a bit - how would you store the data? More importantly: when would you need to? Given this Supervisor structure, the building of our application has changed quit a bit. I don't need to focus on doing small, incremental writes in case our app crashes. Or our database. Or web server for that matter. The only thing that will destroy our Session (and its data) is if the VM crashes or our server loses power. The Erlang VM is not really known to crash - but yes it's a possibility. I certainly don't want to be arrogant about this - but my god - this is too fun. If you had to persist data with only a server crash in mind... how would you do it? How would you compensate for the possible loss of your database - given that you now can? Hopefully you see why I said what I did in the first post in this series about Rails: these two systems are night and day different. See you next time! If you see any corrections or have any thoughts - leave them below. [Less]

When you build applications in the Erlang world you create discrete processes that interact. In theory this is pretty straightforward - until you actually try to do it. Microservices fans out there know the value (and the pain) of managing a fleet of ... [More] services; there are benefits to it, definitely, and also problems. Consider this quote from Benjamin Wootton, describing what he's going through while building a system based on microservices: I am currently involved in architecting a system based around Microservices, and whilst the individual services are very simple, a lot of complexity exists at a higher level level in terms of managing these services and orchestrating business processes throughout them... Microservices [is] one of these ideas that are nice in practice, but all manner of complexity comes out when it meets reality... So very true. I like how Jessica Kerr puts it: Erlang has been providing [the connection between services] for literally 25 years. As we get more and more sophisticated microservice implementations, each one grows their own crappy version of Erlang Erlang has been doing this kind of thing for 25+ years. The community has had time to formalize a number of ideas into concepts, which have made their way into a platform: OTP. That's exactly what we'll be using, right now. The First Task: A Shopping Cart It's a fine place to start, why not. The first thing to do is embrace that I'm building a process here, not a set of objects (Cart and CartItem e.g.). If you think about Shopping as a process that your customer engages in - well I'd say we have our first Application: cd apps mix new shopping --module Redfour.Shopping * creating README.md * creating .gitignore * creating mix.exs * creating config * creating config/config.exs * creating lib * creating lib/shopping.ex * creating test * creating test/test_helper.exs * creating test/shopping_test.exs Your Mix project was created successfully. You can use "mix" to compile it, test it, and more: cd shopping mix test Run "mix help" for more commands. We've just created an Application, which is a discrete OTP construct: When you have written code implementing some specific functionality you might want to make the code into an application, that is, a component that can be started and stopped as a unit, and which can also be reused in other systems. You can think of Applications as "components" or "beans" or "dlls". This fits perfectly well with what we want - our Shopping application will implement a number of its own child services, and it will be used by our web app as well (which is another Application). Task Two: The Session We've setup the structure of our Shopping process - but how will it be carried out? As with Real Life, Shopping is just a concept for customers walking around your store, picking things off the shelves. Each one of them is Shopping, if you will. If I were to break the process down, it would look like this: A Customer enters the store, starting a shopping session as they browse our catalog and various promotions They put stuff in their cart, remove stuff, and decide whether they are buying something They take their selections to checkout and buy them (hopefully) We record the sale and all the data that went into it (because we tracked them through the store) for reporting purposes Functional programming is all about moving/transforming data through a one or more processes. Looking at this list I would say that a Session moves through a Shopping process. Let's define that! The first thing to do is to create an application for Shopping: cd apps mix new shopping --module Redfour.Shopping The next thing is to create a directory and code file: cd apps/shopping mkdir lib/shopping touch lib/shopping/session.ex Now let's pop some code in: defmodule Redfour.Shopping.Session do defstruct [ domain: nil, id: nil, key: nil, landing: "/", ip: "127.0.0.1", member_id: nil, items: [], logs: [], discounts: [] ] end This struct defines our Session explicitly. The domain of the store (in case we have more than one, which is possible), an id from our database, a key that uniquely identifies it in a business sense, and lists to hold items, logs and any discounts that they use. To have an effective store we need to track everything - and this is a great first start. Task Three: The GenServer Any code that you write in Elixir runs in a singular process. Even if you try to write some Elixir and quickly run it with iex (the Elixir REPL) - it's still given a process. At the most basic level a process can be this: iex(1)> spawn(fn() -> IO.puts "Hello World" end) Hello World #PID<0.62.0> The spawn command tells Elixir to spin off some code and run it separately. The fn() -> IO.puts "Hello World" end bits are just an anonymous function declared inline - that part's not important - what is important is how easy it is to spawn things that run in their own process and just do stuff. Note: These are VM processes, completely contained within the Erlang VM. Not OS processes. We would have a massive hill to climb if we decided to build our Session using nothing but spawn. We would, as Jessica put it above: build our own crappy version of Erlang. Let's jump ahead and just use OTP: defmodule Redfour.Shopping.Session do use GenServer def start_link(%{key: key} = args) do GenServer.start_link(__MODULE__,args, name: key) end defstruct [ domain: nil, id: nil, key: nil, landing: "/", ip: "127.0.0.1", member_id: nil, items: [], logs: [], discounts: [] ] end By implementing use GenServer here, we've made our Session into a formalized OTP Server that can do a number of groovy things - the most important of which is that it can now be supervised. More on that in a minute. No matter how much I write and wave my arms - understanding GenServer takes some actual doing. It honestly isn't that complicated - but the first few times you encounter this little beasty ... well you're on Google non-stop (at least I was). There's a learning curve here but, hopefully, we can learn as we go. We have our struct defined and have made our Session into a formalized GenServer process. I need to make sure this thing can maintain state, which might sound a little odd given that we're using a functional language; the whole idea of which is to remove the notion of mutation and, basically, the changing of state. That's what GenServers do: they give you a way to attach state to a process. We do that by seeding it with some data when it starts, and passing that data back after each call. With start_link we seeded the data with a map: def start_link(%{key: key} = args) do GenServer.start_link(__MODULE__,args, name: key) end This is telling OTP to create a process for us in the VM with the initial data of args (which is a Map) and a name corresponding to some key. Here's how I would use this: iex(1)> {:ok, pid} = Redfour.Shopping.Session.start_link %{key: :my_session} {:ok, #PID<0.93.0>} iex(2)> Redfour.Shopping.Session.do_something :my_session, some_arguments We now have a process running in memory that I can refer to anywhere, as long as I know the name :my_session. This process will uniquely describe the notion of someone browsing through our store, looking at things, picking them out, etc. You might be wondering what happens when another customer comes and I need to start another Session? Redfour.Shopping.Session.start_link %{key: :new_customer} Now we have two sessions running. How do I know? Let's have a look: iex(3)> :observer.start You should see an application pop up with all kinds of interesting tabs on it. This is the Erlang observer - it tells you what's happening with the VM and all the processes running inside. If we look for ours: Which is nice and all, but not ideal. Just like any thread or process in other languages, if something goes wrong this process will exit and die - along with all of it's state (items in the cart and so on). This is bad. Speaking of "bad" - you might be wondering about scalability here. How can we expect our app to scale if we have all of these processes bouncing around in memory? This is one of the (many) great parts about the Erlang VM: it's incredibly efficient. Each process has its own heap - which basically means that there's no shared memory between processes. When one dies, everything goes with it. When a parent process dies, all of its children die too. Of course we don't want these processes living forever - so we can actually set a timer on them if we want; but I'd rather handle that explicitly (kiling them) - which I'll do in a later post. Let's flip over to the Applications tab: This tells you what OTP Applications are currently running - and so far it's just one: ours. Process 0.82.0 is our start_link and 0.83.0 is init (which I'll talk about more next time). We then have an Elixir.Logger.Supervisor process which, in turn, has spawned a number of child processes that it needs. Our little processes are nowhere to be found in this tree. We need to change that. Task Four: Supervision In Elixir (and Erlang), you don't write code to recover from errors: You let things die. You do this because you can, and it's a natural thing to do if you want a system that stays up and doesn't crash. This might sound completely bonkers at first - but hold tight hopefully you'll see what I mean. What we need for our Application is a Supervision Tree. In OTP, a Supervisor is a process that watches other processes and, if they die, it restarts them. Pretty simple to explain, a bit harder to correctly put together. When I created our Application I made it rather bare bones with no notion of a Supervisor. If I would have added the --sup flag it would have created a supervisor for us - but that's OK we can just do it ourselves. The first thing to do is to change our apps/shopping/lib/shopping.ex main file: defmodule Redfour.Shopping do use Application import Supervisor.Spec, warn: false def start(_type, _args) do #supervision goes here end end Next, we need to formalize this as the startup module by telling Mix. Open up apps/shopping/mix.exs and edit the application callback to define the start module: def application do [ applications: [:logger], mod: {Redfour.Shopping, []} #add this ] end This callback is telling Mix what additional applications to start, aside from our own. You can see the :logger here - which is what we saw in the :observer above. In addition to these applications, we want Mix to run our Redfour.Shopping module, passing in an empty argument list. Now, let's flip back over to shopping.ex and add our Supervisor code. I'll do this in two separate functions: defmodule Redfour.Shopping do use Application import Supervisor.Spec, warn: false #the entry point to start our app def start(_type, _args) do #supervision goes here start_session_supervisor end #define our parent supervisor def start_session_supervisor do #spec the session supervisor session_worker = worker(Redfour.Shopping.Session, []) Supervisor.start_link([session_worker], strategy: :simple_one_for_one, name: Redfour.SessionSupervisor) end def start_session(key: key) when is_binary(key), do: raise "Please use an atom key" def start_session(key: key) when is_atom(key) do res = Supervisor.start_child(Redfour.SessionSupervisor, [%{key: key}]) end end This took me a while to figure out - hours to be exact. It's a little impenetrable and rather hard to find resources online to properly explain it all - but I'll do my best. The start method is called by Mix when the app starts up. This, in turn calls start_session_supervisor which defines a worker - a thing which will be supervised. This is just a spec, it's not actually starting up the process. I'm using a :simple_one_for_one strategy which means, basically, "if the worker dies, restart it". Finally I'm giving it the name Redfour.ShoppingSupervisor. Now for the good stuff. I'm declaring a method called start_session that will spawn a worker within the purview of a SessionSupervisor. Every time I call this, the SessionSupervisor will call Redfour.Shopping.Session.start_link %{key: key} - creating our GenServer on the VM. Let's give it a whirl. Navigate into the apps/shopping directory and start things up with iex, making sure to tell it to load up the mix.exs file using -S mix: iex -S mix #... iex(1)> Redfour.Shopping.start_session key: :my_key {:ok, #PID<0.96.0>} Yes! Let's have a look at the observer again - run :observer.start within iex one more time. Don't close iex! You should see this in the Applications tab: Here, 91 and 92 are our application starting up. Then we have our Supervisor, which is a child process of our application and finally our Session, identified by our key my_key. Let's add another session to be sure we've done things correctly. Keeping iex open, start another Session: iex(3)> Redfour.Shopping.start_session key: :another_key {:ok, #PID<0.676.0>} Radical! Now let's prove our theory. Keeping iex open still, let's kill a process off and make sure it gets restarted as we expect: iex(4)> pid = Process.whereis :my_key #PID<0.96.0> iex(5)> Process.exit pid, :kill true iex(6)> Process.whereis :my_key #PID<0.683.0> I can grab a running process by name using Process.whereis and get its pid - here you see it's 0.96.0. I then use Process.exit passing in the pid and the directive to kill it off, which indeed happens. If I ask for the process again - I can see it's running once more with a new pid of 0.683.0! You can confirm that it's been restarted looking back at the :observer. Next Time: Working With Data A Session that's built to stay alive with a unique key assigned. Let your brain wander on that a bit - how would you store the data? More importantly: when would you need to? Given this Supervisor structure, the building of our application has changed quit a bit. I don't need to focus on doing small, incremental writes in case our app crashes. Or our database. Or web server for that matter. The only thing that will destroy our Session (and its data) is if the VM crashes or our server loses power. The Erlang VM is not really known to crash - but yes it's a possibility. I certainly don't want to be arrogant about this - but my god - this is too fun. If you had to persist data with only a server crash in mind... how would you do it? How would you compensate for the possible loss of your database - given that you now can? Hopefully you see why I said what I did in the first post in this series about Rails: these two systems are night and day different. See you next time! If you see any corrections or have any thoughts - leave them below. [Less]

When you build applications in the Erlang world you create discrete processes that interact. In theory this is pretty straightforward - until you actually try to do it. Microservices fans out there know the value (and the pain) of managing a fleet of ... [More] services; there are benefits to it, definitely, and also problems. Consider this quote from Benjamin Wootton, describing what he's going through while building a system based on microservices: I am currently involved in architecting a system based around Microservices, and whilst the individual services are very simple, a lot of complexity exists at a higher level level in terms of managing these services and orchestrating business processes throughout them... Microservices [is] one of these ideas that are nice in practice, but all manner of complexity comes out when it meets reality... So very true. I like how Jessica Kerr puts it: Erlang has been providing [the connection between services] for literally 25 years. As we get more and more sophisticated microservice implementations, each one grows their own crappy version of Erlang Erlang has been doing this kind of thing for 25 years. The community has had time to formalize a number of ideas into concepts, which have made their way into a platform: OTP. That's exactly what we'll be using, right now. The First Task: A Shopping Cart It's a fine place to start, why not. The first thing to do is embrace that I'm building a process here, not a set of objects (Cart and CartItem e.g.). If you think about Shopping as a process that your customer engages in - well I'd say we have our first Application: cd apps mix new shopping --module Redfour.Shopping * creating README.md * creating .gitignore * creating mix.exs * creating config * creating config/config.exs * creating lib * creating lib/shopping.ex * creating test * creating test/test_helper.exs * creating test/shopping_test.exs Your Mix project was created successfully. You can use "mix" to compile it, test it, and more: cd shopping mix test Run "mix help" for more commands. We've just created an Application, which is a discrete OTP construct: When you have written code implementing some specific functionality you might want to make the code into an application, that is, a component that can be started and stopped as a unit, and which can also be reused in other systems. You can think of Applications as "components" or "beans" or "dlls". This fits perfectly well with what we want - our Shopping application will implement a number of its own child services, and it will be used by our web app as well (which is another Application). Task Two: The Session We've setup the structure of our Shopping process - but how will it be carried out? As with Real Life, Shopping is just a concept for customers walking around your store, picking things off the shelves. Each one of them is Shopping, if you will. If I were to break the process down, it would look like this: A Customer enters the store, starting a shopping session as they browse our catalog and various promotions They put stuff in their cart, remove stuff, and decide whether they are buying something They take their selections to checkout and buy them (hopefully) We record the sale and all the data that went into it (because we tracked them through the store) for reporting purposes Functional programming is all about moving/transforming data through a one or more processes. Looking at this list I would say that a Session moves through a Shopping process. Let's define that! The first thing to do is to create an application for Shopping: cd apps mix new shopping --module Redfour.Shopping The next thing is to create a directory and code file: cd apps/shopping mkdir lib/shopping touch lib/shopping/session.ex Now let's pop some code in: defmodule Redfour.Shopping.Session do defstruct [ domain: nil, id: nil, key: nil, landing: "/", ip: "127.0.0.1", member_id: nil, items: [], logs: [], discounts: [] ] end This struct defines our Session explicitly. The domain of the store (in case we have more than one, which is possible), an id from our database, a key that uniquely identifies it in a business sense, and lists to hold items, logs and any discounts that they use. To have an effective store we need to track everything - and this is a great first start. Task Three: The GenServer Any code that you write in Elixir runs in a singular process. Even if you try to write some Elixir and quickly run it with iex (the Elixir REPL) - it's still given a process. At the most basic level a process can be this: iex(1)> spawn(fn() -> IO.puts "Hello World" end) Hello World #PID<0.62.0> The spawn command tells Elixir to spin off some code and run it separately. The fn() -> IO.puts "Hello World" end bits are just an anonymous function declared inline - that part's not important - what is important is how easy it is to spawn things that run in their own process and just do stuff. Note: These are VM processes, completely contained within the Erlang VM. Not OS processes. We would have a massive hill to climb if we decided to build our Session using nothing but spawn. We would, as Jessica put it above: build our own crappy versoin of Erlang. Let's jump ahead and just use OTP: defmodule Redfour.Shopping.Session do use GenServer def start_link(%{key: key} = args) do GenServer.start_link(__MODULE__,args, name: key) end defstruct [ domain: nil, id: nil, key: nil, landing: "/", ip: "127.0.0.1", member_id: nil, items: [], logs: [], discounts: [] ] end By implementing use GenServer here, we've made our Session into a formalized OTP Server that can do a number of groovy things - the most important of which is that it can now be supervised. More on that in a minute. No matter how much I write and wave my arms - understanding GenServer takes some actual doing. It honestly isn't that complicated - but the first few times you encounter this little beasty ... well you're on Google non-stop (at least I was). There's a learning curve here but, hopefully, we can learn as we go. We have our struct defined and have made our Session into a formalized GenServer process. I need to make sure this thing can maintain state, which might sound a little odd given that we're using a functional language; the whole idea of which is to remove the notion of mutation and, basically, the changing of state. That's what GenServers do: they give you a way to attach state to a process. We do that by seeding it with some data when it starts, and passing that data back after each call. With start_link we seeded the data with a map: def start_link(%{key: key} = args) do GenServer.start_link(__MODULE__,args, name: key) end This is telling OTP to create a process for us in the VM with the initial data of args (which is a Map) and a name corresponding to some key. Here's how I would use this: iex(1)> {:ok, pid} = Redfour.Shopping.Session.start_link %{key: :my_session} {:ok, #PID<0.93.0>} iex(2)> Redfour.Shopping.Session.do_something :my_session, some_arguments We now have a process running in memory that I can refer to anywhere, as long as I know the name :my_session. This process will uniquely describe the notion of someone browsing through our store, looking at things, picking them out, etc. You might be wondering what happens when another customer comes and I need to start another Session? Redfour.Shopping.Session.start_link %{key: :new_customer} Now we have two sessions running. How do I know? Let's have a look: iex(3)> :observer.start You should see an application pop up with all kinds of interesting tabs on it. This is the Erlang observer - it tells you what's happening with the VM and all the processes running inside. If we look for ours: Which is nice and all, but not ideal. Just like any thread or process in other languages, if something goes wrong this process will exit and die - along with all of it's state (items in the cart and so on). This is bad. Speaking of "bad" - you might be wondering about scalability here. How can we expect our app to scale if we have all of these processes bouncing around in memory? This is one of the (many) great parts about the Erlang VM: it's incredibly efficient. Each process has its own heap - which basically means that there's no shared memory between processes. When one dies, everything goes with it. When a parent process dies, all of its children die too. Of course we don't want these processes living forever - so we can actually set a timer on them if we want; but I'd rather handle that explicitly (kiling them) - which I'll do in a later post. Let's flip over to the Applications tab: This tells you what OTP Applications are currently running - and so far it's just one: ours. Process 0.82.0 is our start_link and 0.83.0 is init (which I'll talk about more next time). We then have an Elixir.Logger.Supervisor process which, in turn, has spawned a number of child processes that it needs. Our little processes are nowhere to be found in this tree. We need to change that. Task Four: Supervision In Elixir (and Erlang), you don't write code to recover from errors: You let things die. You do this because you can, and it's a natural thing to do if you want a system that stays up and doesn't crash. This might sound completely bonkers at first - but hold tight hopefully you'll see what I mean. What we need for our Application is a Supervision Tree. In OTP, a Supervisor is a process that watches other processes and, if they die, it restarts them. Pretty simple to explain, a bit harder to correctly put together. When I created our Application I made it rather bare bones with no notion of a Supervisor. If I would have added the --sup flag it would have created a supervisor for us - but that's OK we can just do it ourselves. The first thing to do is to change our apps/shopping/lib/shopping.ex main file: defmodule Redfour.Shopping do use Application import Supervisor.Spec, warn: false def start(_type, _args) do #supervision goes here end end Next, we need to formalize this as the startup module by telling Mix. Open up apps/lib/shopping/mix.exs and edit the application callback to define the start module: def application do [ applications: [:logger], mod: {Redfour.Shopping, []} #add this ] end THis callback is telling Mix what additional applications to start, aside from our own. You can see the :logger here - which is what we saw in the :observer above. In addition to these applications, we want Mix to run our Redfour.Shopping module, passing in an empty argument list. Now, let's flip back over to session.ex and add our Supervisor code. I'll do this in two separate functions: defmodule Redfour.Shopping do use Application import Supervisor.Spec, warn: false #the entry point to start our app def start(_type, _args) do #supervision goes here start_session_supervisor end #define our parent supervisor def start_session_supervisor do #spec the session supervisor session_worker = worker(Redfour.Shopping.Session, []) Supervisor.start_link([session_worker], strategy: :simple_one_for_one, name: Redfour.SessionSupervisor) end def start_session(key: key) when is_binary(key), do: raise "Please use an atom key" def start_session(key: key) when is_atom(key) do res = Supervisor.start_child(Redfour.SessionSupervisor, [%{key: key}]) end end This took me a while to figure out - hours to be exact. It's a little impenetrable and rather hard to find resources online to properly explain it all - but I'll do my best. The start method is called by Mix when the app starts up. This, in turn calls start_session_supervisor which defines a worker - a thing which will be supervised. This is just a spec, it's not actually starting up the process. I'm using a :simple_one_for_one strategy which means, basically, "if the worker dies, restart it". Finally I'm giving it the name Redfour.ShoppingSupervisor. Now for the good stuff. I'm declaring a method called start_session that will spawn a worker within the purview of a SessionSupervisor. Every time I call this, the SessionSupervisor will call Redfour.Session.start_link %{key: key} - creating our GenServer on the VM. Let's give it a whirl. Navigate into the apps/lib/session directory and start things up with iex, making sure to tell it to load up the mix.exs file using -S mix: iex -S mix #... iex(1)> Redfour.Shopping.start_session key: :my_key {:ok, #PID<0.96.0>} Yes! Let's have a look at the observer again - run :observer.start within iex one more time. Don't close iex! You should see this in the Applications tab: Here, 91 and 92 are our application starting up. Then we have our Supervisor, which is a child process of our application and finally our Session, identified by our key my_key. Let's add another session to be sure we've done things correctly. Keeping iex open, start another Session: iex(3)> Redfour.Shopping.start_session key: :another_key {:ok, #PID<0.676.0>} Radical! Now let's prove our theory. Keeping iex open still, let's kill a process off and make sure it gets restarted as we expect: iex(4)> pid = Process.whereis :my_key #PID<0.96.0> iex(5)> Process.exit pid, :kill true iex(6)> Process.whereis :my_key #PID<0.683.0> I can grab a running process by name using Process.whereis and get its pid - here you see it's 0.96.0. I then use Process.exit passing in the pid and the directive to kill it off, which indeed happens. If I ask for the process again - I can see it's running once more with a new pid of 0.683.0! You can confirm that it's been restarted looking back at the :observer. Next Time: Working With Data A Session that's built to stay alive with a unique key assigned. Let your brain wander on that a bit - how would you store the data? More importantly: when would you need to? Given this Supervisor structure, the building of our application has changed quit a bit. I don't need to focus on doing small, incremental writes in case our app crashes. Or our database. Or web server for that matter. The only thing that will destroy our Session (and its data) is if the VM crashes or our server loses power. The Erlang VM is not really known to crash - but yes it's a possibility. I certainly don't want to be arrogant about this - but my god - this is too fun. If you had to persist data with only a server crash in mind... how would you do it? How would you compensate for the possible loss of your database - given that you now can? Hopefully you see why I said what I did in the first post in this series about Rails: these two systems are night and day different. See you next time! If you see any corrections or have any thoughts - leave them below. [Less]

When you build applications in the Erlang world you create discrete processes that interact. In theory this is pretty straightforward - until you actually try to do it. Microservices fans out there know the value (and the pain) of managing a fleet of ... [More] services; there are benefits to it, definitely, and also problems. Consider this quote from Benjamin Wootton, describing what he's going through while building a system based on microservices: I am currently involved in architecting a system based around Microservices, and whilst the individual services are very simple, a lot of complexity exists at a higher level level in terms of managing these services and orchestrating business processes throughout them... Microservices [is] one of these ideas that are nice in practice, but all manner of complexity comes out when it meets reality... So very true. I like how Jessica Kerr puts it: Erlang has been providing [the connection between services] for literally 25 years. As we get more and more sophisticated microservice implementations, each one grows their own crappy version of Erlang Erlang has been doing this kind of thing for 25+ years. The community has had time to formalize a number of ideas into concepts, which have made their way into a platform: OTP. That's exactly what we'll be using, right now. The First Task: A Shopping Cart It's a fine place to start, why not. The first thing to do is embrace that I'm building a process here, not a set of objects (Cart and CartItem e.g.). If you think about Shopping as a process that your customer engages in - well I'd say we have our first Application: cd apps mix new shopping --module Redfour.Shopping * creating README.md * creating .gitignore * creating mix.exs * creating config * creating config/config.exs * creating lib * creating lib/shopping.ex * creating test * creating test/test_helper.exs * creating test/shopping_test.exs Your Mix project was created successfully. You can use "mix" to compile it, test it, and more: cd shopping mix test Run "mix help" for more commands. We've just created an Application, which is a discrete OTP construct: When you have written code implementing some specific functionality you might want to make the code into an application, that is, a component that can be started and stopped as a unit, and which can also be reused in other systems. You can think of Applications as "components" or "beans" or "dlls". This fits perfectly well with what we want - our Shopping application will implement a number of its own child services, and it will be used by our web app as well (which is another Application). Task Two: The Session We've setup the structure of our Shopping process - but how will it be carried out? As with Real Life, Shopping is just a concept for customers walking around your store, picking things off the shelves. Each one of them is Shopping, if you will. If I were to break the process down, it would look like this: A Customer enters the store, starting a shopping session as they browse our catalog and various promotions They put stuff in their cart, remove stuff, and decide whether they are buying something They take their selections to checkout and buy them (hopefully) We record the sale and all the data that went into it (because we tracked them through the store) for reporting purposes Functional programming is all about moving/transforming data through a one or more processes. Looking at this list I would say that a Session moves through a Shopping process. Let's define that! The first thing to do is to create an application for Shopping: cd apps mix new shopping --module Redfour.Shopping The next thing is to create a directory and code file: cd apps/shopping mkdir lib/shopping touch lib/shopping/session.ex Now let's pop some code in: defmodule Redfour.Shopping.Session do defstruct [ domain: nil, id: nil, key: nil, landing: "/", ip: "127.0.0.1", member_id: nil, items: [], logs: [], discounts: [] ] end This struct defines our Session explicitly. The domain of the store (in case we have more than one, which is possible), an id from our database, a key that uniquely identifies it in a business sense, and lists to hold items, logs and any discounts that they use. To have an effective store we need to track everything - and this is a great first start. Task Three: The GenServer Any code that you write in Elixir runs in a singular process. Even if you try to write some Elixir and quickly run it with iex (the Elixir REPL) - it's still given a process. At the most basic level a process can be this: iex(1)> spawn(fn() -> IO.puts "Hello World" end) Hello World #PID<0.62.0> The spawn command tells Elixir to spin off some code and run it separately. The fn() -> IO.puts "Hello World" end bits are just an anonymous function declared inline - that part's not important - what is important is how easy it is to spawn things that run in their own process and just do stuff. Note: These are VM processes, completely contained within the Erlang VM. Not OS processes. We would have a massive hill to climb if we decided to build our Session using nothing but spawn. We would, as Jessica put it above: build our own crappy version of Erlang. Let's jump ahead and just use OTP: defmodule Redfour.Shopping.Session do use GenServer def start_link(%{key: key} = args) do GenServer.start_link(__MODULE__,args, name: key) end defstruct [ domain: nil, id: nil, key: nil, landing: "/", ip: "127.0.0.1", member_id: nil, items: [], logs: [], discounts: [] ] end By implementing use GenServer here, we've made our Session into a formalized OTP Server that can do a number of groovy things - the most important of which is that it can now be supervised. More on that in a minute. No matter how much I write and wave my arms - understanding GenServer takes some actual doing. It honestly isn't that complicated - but the first few times you encounter this little beasty ... well you're on Google non-stop (at least I was). There's a learning curve here but, hopefully, we can learn as we go. We have our struct defined and have made our Session into a formalized GenServer process. I need to make sure this thing can maintain state, which might sound a little odd given that we're using a functional language; the whole idea of which is to remove the notion of mutation and, basically, the changing of state. That's what GenServers do: they give you a way to attach state to a process. We do that by seeding it with some data when it starts, and passing that data back after each call. With start_link we seeded the data with a map: def start_link(%{key: key} = args) do GenServer.start_link(__MODULE__,args, name: key) end This is telling OTP to create a process for us in the VM with the initial data of args (which is a Map) and a name corresponding to some key. Here's how I would use this: iex(1)> {:ok, pid} = Redfour.Shopping.Session.start_link %{key: :my_session} {:ok, #PID<0.93.0>} iex(2)> Redfour.Shopping.Session.do_something :my_session, some_arguments We now have a process running in memory that I can refer to anywhere, as long as I know the name :my_session. This process will uniquely describe the notion of someone browsing through our store, looking at things, picking them out, etc. You might be wondering what happens when another customer comes and I need to start another Session? Redfour.Shopping.Session.start_link %{key: :new_customer} Now we have two sessions running. How do I know? Let's have a look: iex(3)> :observer.start You should see an application pop up with all kinds of interesting tabs on it. This is the Erlang observer - it tells you what's happening with the VM and all the processes running inside. If we look for ours: Which is nice and all, but not ideal. Just like any thread or process in other languages, if something goes wrong this process will exit and die - along with all of it's state (items in the cart and so on). This is bad. Speaking of "bad" - you might be wondering about scalability here. How can we expect our app to scale if we have all of these processes bouncing around in memory? This is one of the (many) great parts about the Erlang VM: it's incredibly efficient. Each process has its own heap - which basically means that there's no shared memory between processes. When one dies, everything goes with it. When a parent process dies, all of its children die too. Of course we don't want these processes living forever - so we can actually set a timer on them if we want; but I'd rather handle that explicitly (kiling them) - which I'll do in a later post. Let's flip over to the Applications tab: This tells you what OTP Applications are currently running - and so far it's just one: ours. Process 0.82.0 is our start_link and 0.83.0 is init (which I'll talk about more next time). We then have an Elixir.Logger.Supervisor process which, in turn, has spawned a number of child processes that it needs. Our little processes are nowhere to be found in this tree. We need to change that. Task Four: Supervision In Elixir (and Erlang), you don't write code to recover from errors: You let things die. You do this because you can, and it's a natural thing to do if you want a system that stays up and doesn't crash. This might sound completely bonkers at first - but hold tight hopefully you'll see what I mean. What we need for our Application is a Supervision Tree. In OTP, a Supervisor is a process that watches other processes and, if they die, it restarts them. Pretty simple to explain, a bit harder to correctly put together. When I created our Application I made it rather bare bones with no notion of a Supervisor. If I would have added the --sup flag it would have created a supervisor for us - but that's OK we can just do it ourselves. The first thing to do is to change our apps/shopping/lib/shopping.ex main file: defmodule Redfour.Shopping do use Application import Supervisor.Spec, warn: false def start(_type, _args) do #supervision goes here end end Next, we need to formalize this as the startup module by telling Mix. Open up apps/shopping/mix.exs and edit the application callback to define the start module: def application do [ applications: [:logger], mod: {Redfour.Shopping, []} #add this ] end This callback is telling Mix what additional applications to start, aside from our own. You can see the :logger here - which is what we saw in the :observer above. In addition to these applications, we want Mix to run our Redfour.Shopping module, passing in an empty argument list. Now, let's flip back over to shopping.ex and add our Supervisor code. I'll do this in two separate functions: defmodule Redfour.Shopping do use Application import Supervisor.Spec, warn: false #the entry point to start our app def start(_type, _args) do #supervision goes here start_session_supervisor end #define our parent supervisor def start_session_supervisor do #spec the session supervisor session_worker = worker(Redfour.Shopping.Session, []) Supervisor.start_link([session_worker], strategy: :simple_one_for_one, name: Redfour.SessionSupervisor) end def start_session(key: key) when is_binary(key), do: raise "Please use an atom key" def start_session(key: key) when is_atom(key) do res = Supervisor.start_child(Redfour.SessionSupervisor, [%{key: key}]) end end This took me a while to figure out - hours to be exact. It's a little impenetrable and rather hard to find resources online to properly explain it all - but I'll do my best. The start method is called by Mix when the app starts up. This, in turn calls start_session_supervisor which defines a worker - a thing which will be supervised. This is just a spec, it's not actually starting up the process. I'm using a :simple_one_for_one strategy which means, basically, "if the worker dies, restart it". Finally I'm giving it the name Redfour.ShoppingSupervisor. Now for the good stuff. I'm declaring a method called start_session that will spawn a worker within the purview of a SessionSupervisor. Every time I call this, the SessionSupervisor will call Redfour.Shopping.Session.start_link %{key: key} - creating our GenServer on the VM. Let's give it a whirl. Navigate into the apps/shopping directory and start things up with iex, making sure to tell it to load up the mix.exs file using -S mix: iex -S mix #... iex(1)> Redfour.Shopping.start_session key: :my_key {:ok, #PID<0.96.0>} Yes! Let's have a look at the observer again - run :observer.start within iex one more time. Don't close iex! You should see this in the Applications tab: Here, 91 and 92 are our application starting up. Then we have our Supervisor, which is a child process of our application and finally our Session, identified by our key my_key. Let's add another session to be sure we've done things correctly. Keeping iex open, start another Session: iex(3)> Redfour.Shopping.start_session key: :another_key {:ok, #PID<0.676.0>} Radical! Now let's prove our theory. Keeping iex open still, let's kill a process off and make sure it gets restarted as we expect: iex(4)> pid = Process.whereis :my_key #PID<0.96.0> iex(5)> Process.exit pid, :kill true iex(6)> Process.whereis :my_key #PID<0.683.0> I can grab a running process by name using Process.whereis and get its pid - here you see it's 0.96.0. I then use Process.exit passing in the pid and the directive to kill it off, which indeed happens. If I ask for the process again - I can see it's running once more with a new pid of 0.683.0! You can confirm that it's been restarted looking back at the :observer. Next Time: Working With Data A Session that's built to stay alive with a unique key assigned. Let your brain wander on that a bit - how would you store the data? More importantly: when would you need to? Given this Supervisor structure, the building of our application has changed quit a bit. I don't need to focus on doing small, incremental writes in case our app crashes. Or our database. Or web server for that matter. The only thing that will destroy our Session (and its data) is if the VM crashes or our server loses power. The Erlang VM is not really known to crash - but yes it's a possibility. I certainly don't want to be arrogant about this - but my god - this is too fun. If you had to persist data with only a server crash in mind... how would you do it? How would you compensate for the possible loss of your database - given that you now can? Hopefully you see why I said what I did in the first post in this series about Rails: these two systems are night and day different. See you next time! If you see any corrections or have any thoughts - leave them below. Want to learn Elixir? Learn how to build fast, fault-tolerant applications with Elixir. This is not a traditional, boring tutorial. You'll get an ebook (epub or mobi) as well as 3 hours worth of tightly-edited, lovingly produced Elixir content. You'll learn Elixir while doing Elixir, helping me out at my new fictional job as development lead at Red:4 Aerospace. [Less]

When you build applications in the Erlang world you create discrete processes that interact. In theory this is pretty straightforward - until you actually try to do it. Microservices fans out there know the value (and the pain) of managing a fleet of ... [More] services; there are benefits to it, definitely, and also problems. Consider this quote from Benjamin Wootton, describing what he's going through while building a system based on microservices: I am currently involved in architecting a system based around Microservices, and whilst the individual services are very simple, a lot of complexity exists at a higher level level in terms of managing these services and orchestrating business processes throughout them... Microservices [is] one of these ideas that are nice in practice, but all manner of complexity comes out when it meets reality... So very true. I like how Jessica Kerr puts it: Erlang has been providing [the connection between services] for literally 25 years. As we get more and more sophisticated microservice implementations, each one grows their own crappy version of Erlang Erlang has been doing this kind of thing for 25 years. The community has had time to formalize a number of ideas into concepts, which have made their way into a platform: OTP. That's exactly what we'll be using, right now. The First Task: A Shopping Cart It's a fine place to start, why not. The first thing to do is embrace that I'm building a process here, not a set of objects (Cart and CartItem e.g.). If you think about Shopping as a process that your customer engages in - well I'd say we have our first Application: cd apps mix new shopping --module Redfour.Shopping * creating README.md * creating .gitignore * creating mix.exs * creating config * creating config/config.exs * creating lib * creating lib/shopping.ex * creating test * creating test/test_helper.exs * creating test/shopping_test.exs Your Mix project was created successfully. You can use "mix" to compile it, test it, and more: cd shopping mix test Run "mix help" for more commands. We've just created an Application, which is a discrete OTP construct: When you have written code implementing some specific functionality you might want to make the code into an application, that is, a component that can be started and stopped as a unit, and which can also be reused in other systems. You can think of Applications as "components" or "beans" or "dlls". This fits perfectly well with what we want - our Shopping application will implement a number of its own child services, and it will be used by our web app as well (which is another Application). Task Two: The Session We've setup the structure of our Shopping process - but how will it be carried out? As with Real Life, Shopping is just a concept for customers walking around your store, picking things off the shelves. Each one of them is Shopping, if you will. If I were to break the process down, it would look like this: A Customer enters the store, starting a shopping session as they browse our catalog and various promotions They put stuff in their cart, remove stuff, and decide whether they are buying something They take their selections to checkout and buy them (hopefully) We record the sale and all the data that went into it (because we tracked them through the store) for reporting purposes Functional programming is all about moving/transforming data through a one or more processes. Looking at this list I would say that a Session moves through a Shopping process. Let's define that! The first thing to do is to create an application for Shopping: cd apps mix new shopping --module Redfour.Shopping The next thing is to create a directory and code file: cd apps/shopping mkdir lib/shopping touch lib/shopping/session.ex Now let's pop some code in: defmodule Redfour.Shopping.Session do defstruct [ domain: nil, id: nil, key: nil, landing: "/", ip: "127.0.0.1", member_id: nil, items: [], logs: [], discounts: [] ] end This struct defines our Session explicitly. The domain of the store (in case we have more than one, which is possible), an id from our database, a key that uniquely identifies it in a business sense, and lists to hold items, logs and any discounts that they use. To have an effective store we need to track everything - and this is a great first start. Task Three: The GenServer Any code that you write in Elixir runs in a singular process. Even if you try to write some Elixir and quickly run it with iex (the Elixir REPL) - it's still given a process. At the most basic level a process can be this: iex(1)> spawn(fn() -> IO.puts "Hello World" end) Hello World #PID<0.62.0> The spawn command tells Elixir to spin off some code and run it separately. The fn() -> IO.puts "Hello World" end bits are just an anonymous function declared inline - that part's not important - what is important is how easy it is to spawn things that run in their own process and just do stuff. Note: These are VM processes, completely contained within the Erlang VM. Not OS processes. We would have a massive hill to climb if we decided to build our Session using nothing but spawn. We would, as Jessica put it above: build our own crappy version of Erlang. Let's jump ahead and just use OTP: defmodule Redfour.Shopping.Session do use GenServer def start_link(%{key: key} = args) do GenServer.start_link(__MODULE__,args, name: key) end defstruct [ domain: nil, id: nil, key: nil, landing: "/", ip: "127.0.0.1", member_id: nil, items: [], logs: [], discounts: [] ] end By implementing use GenServer here, we've made our Session into a formalized OTP Server that can do a number of groovy things - the most important of which is that it can now be supervised. More on that in a minute. No matter how much I write and wave my arms - understanding GenServer takes some actual doing. It honestly isn't that complicated - but the first few times you encounter this little beasty ... well you're on Google non-stop (at least I was). There's a learning curve here but, hopefully, we can learn as we go. We have our struct defined and have made our Session into a formalized GenServer process. I need to make sure this thing can maintain state, which might sound a little odd given that we're using a functional language; the whole idea of which is to remove the notion of mutation and, basically, the changing of state. That's what GenServers do: they give you a way to attach state to a process. We do that by seeding it with some data when it starts, and passing that data back after each call. With start_link we seeded the data with a map: def start_link(%{key: key} = args) do GenServer.start_link(__MODULE__,args, name: key) end This is telling OTP to create a process for us in the VM with the initial data of args (which is a Map) and a name corresponding to some key. Here's how I would use this: iex(1)> {:ok, pid} = Redfour.Shopping.Session.start_link %{key: :my_session} {:ok, #PID<0.93.0>} iex(2)> Redfour.Shopping.Session.do_something :my_session, some_arguments We now have a process running in memory that I can refer to anywhere, as long as I know the name :my_session. This process will uniquely describe the notion of someone browsing through our store, looking at things, picking them out, etc. You might be wondering what happens when another customer comes and I need to start another Session? Redfour.Shopping.Session.start_link %{key: :new_customer} Now we have two sessions running. How do I know? Let's have a look: iex(3)> :observer.start You should see an application pop up with all kinds of interesting tabs on it. This is the Erlang observer - it tells you what's happening with the VM and all the processes running inside. If we look for ours: Which is nice and all, but not ideal. Just like any thread or process in other languages, if something goes wrong this process will exit and die - along with all of it's state (items in the cart and so on). This is bad. Speaking of "bad" - you might be wondering about scalability here. How can we expect our app to scale if we have all of these processes bouncing around in memory? This is one of the (many) great parts about the Erlang VM: it's incredibly efficient. Each process has its own heap - which basically means that there's no shared memory between processes. When one dies, everything goes with it. When a parent process dies, all of its children die too. Of course we don't want these processes living forever - so we can actually set a timer on them if we want; but I'd rather handle that explicitly (kiling them) - which I'll do in a later post. Let's flip over to the Applications tab: This tells you what OTP Applications are currently running - and so far it's just one: ours. Process 0.82.0 is our start_link and 0.83.0 is init (which I'll talk about more next time). We then have an Elixir.Logger.Supervisor process which, in turn, has spawned a number of child processes that it needs. Our little processes are nowhere to be found in this tree. We need to change that. Task Four: Supervision In Elixir (and Erlang), you don't write code to recover from errors: You let things die. You do this because you can, and it's a natural thing to do if you want a system that stays up and doesn't crash. This might sound completely bonkers at first - but hold tight hopefully you'll see what I mean. What we need for our Application is a Supervision Tree. In OTP, a Supervisor is a process that watches other processes and, if they die, it restarts them. Pretty simple to explain, a bit harder to correctly put together. When I created our Application I made it rather bare bones with no notion of a Supervisor. If I would have added the --sup flag it would have created a supervisor for us - but that's OK we can just do it ourselves. The first thing to do is to change our apps/shopping/lib/shopping.ex main file: defmodule Redfour.Shopping do use Application import Supervisor.Spec, warn: false def start(_type, _args) do #supervision goes here end end Next, we need to formalize this as the startup module by telling Mix. Open up apps/shopping/mix.exs and edit the application callback to define the start module: def application do [ applications: [:logger], mod: {Redfour.Shopping, []} #add this ] end This callback is telling Mix what additional applications to start, aside from our own. You can see the :logger here - which is what we saw in the :observer above. In addition to these applications, we want Mix to run our Redfour.Shopping module, passing in an empty argument list. Now, let's flip back over to shopping.ex and add our Supervisor code. I'll do this in two separate functions: defmodule Redfour.Shopping do use Application import Supervisor.Spec, warn: false #the entry point to start our app def start(_type, _args) do #supervision goes here start_session_supervisor end #define our parent supervisor def start_session_supervisor do #spec the session supervisor session_worker = worker(Redfour.Shopping.Session, []) Supervisor.start_link([session_worker], strategy: :simple_one_for_one, name: Redfour.SessionSupervisor) end def start_session(key: key) when is_binary(key), do: raise "Please use an atom key" def start_session(key: key) when is_atom(key) do res = Supervisor.start_child(Redfour.SessionSupervisor, [%{key: key}]) end end This took me a while to figure out - hours to be exact. It's a little impenetrable and rather hard to find resources online to properly explain it all - but I'll do my best. The start method is called by Mix when the app starts up. This, in turn calls start_session_supervisor which defines a worker - a thing which will be supervised. This is just a spec, it's not actually starting up the process. I'm using a :simple_one_for_one strategy which means, basically, "if the worker dies, restart it". Finally I'm giving it the name Redfour.ShoppingSupervisor. Now for the good stuff. I'm declaring a method called start_session that will spawn a worker within the purview of a SessionSupervisor. Every time I call this, the SessionSupervisor will call Redfour.Session.start_link %{key: key} - creating our GenServer on the VM. Let's give it a whirl. Navigate into the apps/shopping directory and start things up with iex, making sure to tell it to load up the mix.exs file using -S mix: iex -S mix #... iex(1)> Redfour.Shopping.start_session key: :my_key {:ok, #PID<0.96.0>} Yes! Let's have a look at the observer again - run :observer.start within iex one more time. Don't close iex! You should see this in the Applications tab: Here, 91 and 92 are our application starting up. Then we have our Supervisor, which is a child process of our application and finally our Session, identified by our key my_key. Let's add another session to be sure we've done things correctly. Keeping iex open, start another Session: iex(3)> Redfour.Shopping.start_session key: :another_key {:ok, #PID<0.676.0>} Radical! Now let's prove our theory. Keeping iex open still, let's kill a process off and make sure it gets restarted as we expect: iex(4)> pid = Process.whereis :my_key #PID<0.96.0> iex(5)> Process.exit pid, :kill true iex(6)> Process.whereis :my_key #PID<0.683.0> I can grab a running process by name using Process.whereis and get its pid - here you see it's 0.96.0. I then use Process.exit passing in the pid and the directive to kill it off, which indeed happens. If I ask for the process again - I can see it's running once more with a new pid of 0.683.0! You can confirm that it's been restarted looking back at the :observer. Next Time: Working With Data A Session that's built to stay alive with a unique key assigned. Let your brain wander on that a bit - how would you store the data? More importantly: when would you need to? Given this Supervisor structure, the building of our application has changed quit a bit. I don't need to focus on doing small, incremental writes in case our app crashes. Or our database. Or web server for that matter. The only thing that will destroy our Session (and its data) is if the VM crashes or our server loses power. The Erlang VM is not really known to crash - but yes it's a possibility. I certainly don't want to be arrogant about this - but my god - this is too fun. If you had to persist data with only a server crash in mind... how would you do it? How would you compensate for the possible loss of your database - given that you now can? Hopefully you see why I said what I did in the first post in this series about Rails: these two systems are night and day different. See you next time! If you see any corrections or have any thoughts - leave them below. [Less]

It's time to dive right into the deep end and consider how we're going to use OTP. The learning curve is deceptively steep, but if we can do this correctly we'll have a rather bullet-proof system.

As CTO I get to call the shots here at Red:4 but I do have to answer to the CEO and others. It's easy to arm-wave, to go on and on about Elixir and functional languages - but at the end of the day it's what you do, not what you say that counts. ... [More] Putting Elixir To The Test Given that, I've decided to build an eCommerce store using Elixir and Phoenix. I want to stress the language (and myself) - to solve problems with its constructs and to throw it all at Phoenix, to see how it responds. If you know anything about Elixir than you know it runs on the Erlang VM. That's like telling your friends your mom gave you the jet the for the weekend so you could fly to Burning Man. I suppose that's a bit hyperbolic. In truth I actually don't know first-hand how powerful Erlang is in production short of what I've been told (over and over): Whatsapp (powered by Erlang) sold for 22 billion - mostly for its engineeering, Erlang systems have claimed "nine nines reliability" which means they basically went down for a second or so over the span of 20 years, Phoenix clipped 2 million concurrent sockets on a single server - stuff like that. Neat stories, I want to do more than read. I want to see how the sausage is made and if I, as a really crappy developer, can expect to have this same power at my disposal. First: I expect to fail and fail often. This isn't a tutorial, this isn't me trying to sell you Elixir/Erlang. This is me exploring what can be done with the Elixir/Phoenix platform, to see if I can put together a reasonably complex app, discovering aspects of the language (and OTP) along the way. Some Thoughts At The Outset At the core of it: I'm building a website. If you read over the docs for Phoenix you quickly see something that looks like Rails. If you heard a shout in the distance, that would be Chris McCord and Jose Valim yelling at me. They hate the comparison because they believe (rightfully so) that Phoenix is something quite different than Rails. As much as they might dislike the comparison, there just simply is no escaping that Phoenix, the way its presented, looks like a Rails clone. Yes: a clone. Models, Migrations, Controllers and Views using Elixir's Ruby-inspired syntax. It feels like Rails as much as Jose and Chris want it to be otherwise. I do believe it's a lot more than that - and it's one of the reasons I decided to write this set of posts. There's nothing wrong with Rails and I want to quickly veer away from "Rails vs. Elixir/Phoenix" as fast as I can. You just can't compare these two things; they are, at their core, utterly different even though they look the same. The only thing I'll say about Rails at this point is that I am not embracing the Majestic Monstrosity: The Majestic Monolith is how a small team like Basecamp can do a complete rewrite from scratch in 18 months across all platforms at once.— DHH (@dhh) February 4, 2016 I ran my business on Rails for four years and, like DHH, I rewrote my app because of the improvements to Rails and also because when you try to change anything in a Majestic Monstrosity it tends to fall apart. I'm sure it was me - I suck. Now let's move on. Enough Talking: Mix New Mix is Elixir's workhorse. It's a project tool, your compiler, test-runner, task-runner - it's basically Your Little Elixir Buddy. To create a project you simply: mix new thing ... where thing is the name of your project. Your project can be a big application or a small component. Phoenix builds on this and has a task of its very own: mix phoenix.new redfour Doing this will give me a phoenix site: mix phoenix.new redfour * creating redfour/config/config.exs * creating redfour/config/dev.exs * creating redfour/config/prod.exs * creating redfour/config/prod.secret.exs * creating redfour/config/test.exs * creating redfour/lib/redfour.ex * creating redfour/lib/redfour/endpoint.ex * creating redfour/test/views/error_view_test.exs * creating redfour/test/support/conn_case.ex * creating redfour/test/support/channel_case.ex * creating redfour/test/test_helper.exs * creating redfour/web/channels/user_socket.ex * creating redfour/web/router.ex * creating redfour/web/views/error_view.ex * creating redfour/web/web.ex * creating redfour/mix.exs * creating redfour/README.md * creating redfour/lib/redfour/repo.ex * creating redfour/test/support/model_case.ex * creating redfour/priv/repo/seeds.exs * creating redfour/.gitignore * creating redfour/brunch-config.js * creating redfour/package.json * creating redfour/web/static/css/app.css * creating redfour/web/static/js/app.js * creating redfour/web/static/js/socket.js * creating redfour/web/static/assets/robots.txt * creating redfour/web/static/assets/images/phoenix.png * creating redfour/web/static/assets/favicon.ico * creating redfour/test/controllers/page_controller_test.exs * creating redfour/test/views/layout_view_test.exs * creating redfour/test/views/page_view_test.exs * creating redfour/web/controllers/page_controller.ex * creating redfour/web/templates/layout/app.html.eex * creating redfour/web/templates/page/index.html.eex * creating redfour/web/views/layout_view.ex * creating redfour/web/views/page_view.ex Fetch and install dependencies? [Yn] This task created the skeleton of my web app, including: Tests for controllers, views and layouts which I don't want (the controller test fails when you change the index page template's text. That kind of thing drive me NUTS) A complete Brunch setup for handling my assets for me which is nice An Ecto Repository for data access A web directory, which is where models, controllers, views and channels go A smattering of other things I want about 60% of this. Forgive me - the first bits of this post sound kind of negative; they're not meant to be. I've learned the hard way over the past 10 years (or so) what can happen when you give control of the development process over to a framework and its conventions. These conventions are designed to get you off the ground fast but can (and have) make future maintenance a bit of a problem (see a few paragraphs above RE rewrites). What I want is this: functionality broken down into discrete components and processes that can be tested individually, that are isolated, and do not rely on a global framework to feel and embrace the love of OTP (Elixir/Erlang's underlying framework) at every level to not grow my web application into a monster Unfortunately the Phoenix guides walk you through a process of adding to the monolith - dropping a model and migration here, a controller there, adding a view and a template. There is so much more that's possible. Let's turn this in a positive direction. Creating Our App Directory Our application will be comprised of many parts; one of which is our web site. For this, Elixir's umbrella project structure will be perfect. It creates a structure for you that all your "sub applications" can live in - sharing dependencies while staying relatively independent: mix new redfour --umbrella * creating .gitignore * creating README.md * creating mix.exs * creating apps * creating config * creating config/config.exs Your umbrella project was created successfully. Inside your project, you will find an apps/ directory where you can create and host many apps: cd redfour cd apps mix new my_app Commands like "mix compile" and "mix test" when executed in the umbrella project root will automatically run for each application in the apps/ directory. Perfect. These little "sub applications" are loosely associated in that they share a dependencies folder (not the dependencies themselves - just the folder on disk), you can run your entire project's tests from the root, and you can reference each application using some syntactic sugar. Let's drop into the apps folder and install Phoenix. This time I'm going to strip it down to the bare essentials - I don't want to use an ORM-y database tool (Ecto) and I don't want to use Brunch; I have a way I like to manage/compress assets: cd redfour/apps mix phoenix.new web --no-brunch --no-ecto --module Redfour.Web ... Have a look in your /web directory. It's so trim and clean! Now have a look in the /deps directory in your root: ls -la ../deps drwxr-xr-x 11 rob staff 374 Feb 10 16:25 . drwxr-xr-x 9 rob staff 306 Feb 10 16:25 .. drwxr-xr-x 11 rob staff 374 Feb 10 16:25 cowboy drwxr-xr-x 10 rob staff 340 Feb 10 16:25 cowlib drwxr-xr-x 12 rob staff 408 Feb 10 16:25 fs drwxr-xr-x 14 rob staff 476 Feb 10 16:25 phoenix drwxr-xr-x 13 rob staff 442 Feb 10 16:25 phoenix_html drwxr-xr-x 9 rob staff 306 Feb 10 16:25 phoenix_live_reload drwxr-xr-x 9 rob staff 306 Feb 10 16:25 plug drwxr-xr-x 10 rob staff 340 Feb 10 16:25 poison drwxr-xr-x 9 rob staff 306 Feb 10 16:25 ranch A central place with all the dependencies for your project so you don't have bloat and dependency repetition. Ahhhhhhhhh. This is lovely. A stripped-down version of this radical framework that I can tweak as I need to. I like Controllers, I like the idea of Views (which I'll talk about more later on) and the routes/templating are great. There is so much goodness here, which I really feel is clouded a bit by the way it resembles Rails. Next Up: Our First Process We have our skeleton and we're ready to rock. As opposed to deciding up-front what data access I want, logging, infrastructure and everything else, I'm going to lean on OTP and think about things in terms of processes and (to a degree) monitored services. How would you do this for a commerce app? Let's see if our opinions match up for the next post. Feel free to leave a comment below. [Less]

As CTO I get to call the shots here at Red:4 but I do have to answer to the CEO and others. It's easy to arm-wave, to go on and on about Elixir and functional languages - but at the end of the day it's what you do, not what you say that counts. ... [More] Putting Elixir To The Test Given that, I've decided to build an eCommerce store using Elixir and Phoenix. I want to stress the language (and myself) - to solve problems with its constructs and to throw it all at Phoenix, to see how it responds. If you know anything about Elixir then you know it runs on the Erlang VM. That's like telling your friends your mom gave you the jet for the weekend so you could fly to Burning Man. I suppose that's a bit hyperbolic. In truth I actually don't know first-hand how powerful Erlang is in production short of what I've been told (over and over): Whatsapp (powered by Erlang) sold for 22 billion - mostly for its engineeering, Erlang systems have claimed "nine nines reliability" which means they basically went down for a second or so over the span of 20 years, Phoenix clipped 2 million concurrent sockets on a single server - stuff like that. Neat stories, I want to do more than read. I want to see how the sausage is made and if I, as a really crappy developer, can expect to have this same power at my disposal. First: I expect to fail and fail often. This isn't a tutorial, this isn't me trying to sell you Elixir/Erlang. This is me exploring what can be done with the Elixir/Phoenix platform, to see if I can put together a reasonably complex app, discovering aspects of the language (and OTP) along the way. Some Thoughts At The Outset At the core of it: I'm building a website. If you read over the docs for Phoenix you quickly see something that looks like Rails. If you heard a shout in the distance, that would be Chris McCord and Jose Valim yelling at me. They hate the comparison because they believe (rightfully so) that Phoenix is something quite different than Rails. As much as they might dislike the comparison, there just simply is no escaping that Phoenix, the way its presented, looks like a Rails clone. Yes: a clone. Models, Migrations, Controllers and Views using Elixir's Ruby-inspired syntax. It feels like Rails as much as Jose and Chris want it to be otherwise. I do believe it's a lot more than that - and it's one of the reasons I decided to write this set of posts. There's nothing wrong with Rails and I want to quickly veer away from "Rails vs. Elixir/Phoenix" as fast as I can. You just can't compare these two things; they are, at their core, utterly different even though they look the same. The only thing I'll say about Rails at this point is that I am not embracing the Majestic Monstrosity: The Majestic Monolith is how a small team like Basecamp can do a complete rewrite from scratch in 18 months across all platforms at once.— DHH (@dhh) February 4, 2016 I ran my business on Rails for four years and, like DHH, I rewrote my app because of the improvements to Rails and also because when you try to change anything in a Majestic Monstrosity it tends to fall apart. I'm sure it was me - I suck. Now let's move on. Enough Talking: Mix New Mix is Elixir's workhorse. It's a project tool, your compiler, test-runner, task-runner - it's basically Your Little Elixir Buddy. To create a project you simply: mix new thing ... where thing is the name of your project. Your project can be a big application or a small component. Phoenix builds on this and has a task of its very own: mix phoenix.new redfour Doing this will give me a phoenix site: mix phoenix.new redfour * creating redfour/config/config.exs * creating redfour/config/dev.exs * creating redfour/config/prod.exs * creating redfour/config/prod.secret.exs * creating redfour/config/test.exs * creating redfour/lib/redfour.ex * creating redfour/lib/redfour/endpoint.ex * creating redfour/test/views/error_view_test.exs * creating redfour/test/support/conn_case.ex * creating redfour/test/support/channel_case.ex * creating redfour/test/test_helper.exs * creating redfour/web/channels/user_socket.ex * creating redfour/web/router.ex * creating redfour/web/views/error_view.ex * creating redfour/web/web.ex * creating redfour/mix.exs * creating redfour/README.md * creating redfour/lib/redfour/repo.ex * creating redfour/test/support/model_case.ex * creating redfour/priv/repo/seeds.exs * creating redfour/.gitignore * creating redfour/brunch-config.js * creating redfour/package.json * creating redfour/web/static/css/app.css * creating redfour/web/static/js/app.js * creating redfour/web/static/js/socket.js * creating redfour/web/static/assets/robots.txt * creating redfour/web/static/assets/images/phoenix.png * creating redfour/web/static/assets/favicon.ico * creating redfour/test/controllers/page_controller_test.exs * creating redfour/test/views/layout_view_test.exs * creating redfour/test/views/page_view_test.exs * creating redfour/web/controllers/page_controller.ex * creating redfour/web/templates/layout/app.html.eex * creating redfour/web/templates/page/index.html.eex * creating redfour/web/views/layout_view.ex * creating redfour/web/views/page_view.ex Fetch and install dependencies? [Yn] This task created the skeleton of my web app, including: Tests for controllers, views and layouts which I don't want (the controller test fails when you change the index page template's text. That kind of thing drive me NUTS) A complete Brunch setup for handling my assets for me which is nice An Ecto Repository for data access A web directory, which is where models, controllers, views and channels go A smattering of other things I want about 60% of this. Forgive me - the first bits of this post sound kind of negative; they're not meant to be. I've learned the hard way over the past 10 years (or so) what can happen when you give control of the development process over to a framework and its conventions. These conventions are designed to get you off the ground fast but can (and have) make future maintenance a bit of a problem (see a few paragraphs above RE rewrites). What I want is this: functionality broken down into discrete components and processes that can be tested individually, that are isolated, and do not rely on a global framework to feel and embrace the love of OTP (Elixir/Erlang's underlying framework) at every level to not grow my web application into a monster Unfortunately the Phoenix guides walk you through a process of adding to the monolith - dropping a model and migration here, a controller there, adding a view and a template. There is so much more that's possible. Let's turn this in a positive direction. Creating Our App Directory Our application will be comprised of many parts; one of which is our web site. For this, Elixir's umbrella project structure will be perfect. It creates a structure for you that all your "sub applications" can live in - sharing dependencies while staying relatively independent: mix new redfour --umbrella * creating .gitignore * creating README.md * creating mix.exs * creating apps * creating config * creating config/config.exs Your umbrella project was created successfully. Inside your project, you will find an apps/ directory where you can create and host many apps: cd redfour cd apps mix new my_app Commands like "mix compile" and "mix test" when executed in the umbrella project root will automatically run for each application in the apps/ directory. Perfect. These little "sub applications" are loosely associated in that they share a dependencies folder (not the dependencies themselves - just the folder on disk), you can run your entire project's tests from the root, and you can reference each application using some syntactic sugar. Let's drop into the apps folder and install Phoenix. This time I'm going to strip it down to the bare essentials - I don't want to use an ORM-y database tool (Ecto) and I don't want to use Brunch; I have a way I like to manage/compress assets: cd redfour/apps mix phoenix.new web --no-brunch --no-ecto --module Redfour.Web ... Have a look in your /web directory. It's so trim and clean! Now have a look in the /deps directory in your root: ls -la ../deps drwxr-xr-x 11 rob staff 374 Feb 10 16:25 . drwxr-xr-x 9 rob staff 306 Feb 10 16:25 .. drwxr-xr-x 11 rob staff 374 Feb 10 16:25 cowboy drwxr-xr-x 10 rob staff 340 Feb 10 16:25 cowlib drwxr-xr-x 12 rob staff 408 Feb 10 16:25 fs drwxr-xr-x 14 rob staff 476 Feb 10 16:25 phoenix drwxr-xr-x 13 rob staff 442 Feb 10 16:25 phoenix_html drwxr-xr-x 9 rob staff 306 Feb 10 16:25 phoenix_live_reload drwxr-xr-x 9 rob staff 306 Feb 10 16:25 plug drwxr-xr-x 10 rob staff 340 Feb 10 16:25 poison drwxr-xr-x 9 rob staff 306 Feb 10 16:25 ranch A central place with all the dependencies for your project so you don't have bloat and dependency repetition. Ahhhhhhhhh. This is lovely. A stripped-down version of this radical framework that I can tweak as I need to. I like Controllers, I like the idea of Views (which I'll talk about more later on) and the routes/templating are great. There is so much goodness here, which I really feel is clouded a bit by the way it resembles Rails. Next Up: Our First Process We have our skeleton and we're ready to rock. As opposed to deciding up-front what data access I want, logging, infrastructure and everything else, I'm going to lean on OTP and think about things in terms of processes and (to a degree) monitored services. How would you do this for a commerce app? Let's see if our opinions match up for the next post. Feel free to leave a comment below. [Less]

As CTO I get to call the shots here at Red:4 but I do have to answer to the CEO and others. It's easy to arm-wave, to go on and on about Elixir and functional languages - but at the end of the day it's what you do, not what you say that counts. ... [More] Putting Elixir To The Test Given that, I've decided to build an eCommerce store using Elixir and Phoenix. I want to stress the language (and myself) - to solve problems with its constructs and to throw it all at Phoenix, to see how it responds. If you know anything about Elixir then you know it runs on the Erlang VM. That's like telling your friends your mom gave you the jet for the weekend so you could fly to Burning Man. I suppose that's a bit hyperbolic. In truth I actually don't know first-hand how powerful Erlang is in production short of what I've been told (over and over): Whatsapp (powered by Erlang) sold for 22 billion - mostly for its engineeering, Erlang systems have claimed "nine nines reliability" which means they basically went down for a second or so over the span of 20 years, Phoenix clipped 2 million concurrent sockets on a single server - stuff like that. Neat stories, I want to do more than read. I want to see how the sausage is made and if I, as a really crappy developer, can expect to have this same power at my disposal. First: I expect to fail and fail often. This isn't a tutorial, this isn't me trying to sell you Elixir/Erlang. This is me exploring what can be done with the Elixir/Phoenix platform, to see if I can put together a reasonably complex app, discovering aspects of the language (and OTP) along the way. Some Thoughts At The Outset At the core of it: I'm building a website. If you read over the docs for Phoenix you quickly see something that looks like Rails. If you heard a shout in the distance, that would be Chris McCord and Jose Valim yelling at me. They hate the comparison because they believe (rightfully so) that Phoenix is something quite different than Rails. As much as they might dislike the comparison, there just simply is no escaping that Phoenix, the way its presented, looks like a Rails clone. Yes: a clone. Models, Migrations, Controllers and Views using Elixir's Ruby-inspired syntax. It feels like Rails as much as Jose and Chris want it to be otherwise. I do believe it's a lot more than that - and it's one of the reasons I decided to write this set of posts. There's nothing wrong with Rails and I want to quickly veer away from "Rails vs. Elixir/Phoenix" as fast as I can. You just can't compare these two things; they are, at their core, utterly different even though they look the same. The only thing I'll say about Rails at this point is that I am not embracing the Majestic Monstrosity: The Majestic Monolith is how a small team like Basecamp can do a complete rewrite from scratch in 18 months across all platforms at once.— DHH (@dhh) February 4, 2016 I ran my business on Rails for four years and, like DHH, I rewrote my app because of the improvements to Rails and also because when you try to change anything in a Majestic Monstrosity it tends to fall apart. I'm sure it was me - I suck. Now let's move on. Enough Talking: Mix New Mix is Elixir's workhorse. It's a project tool, your compiler, test-runner, task-runner - it's basically Your Little Elixir Buddy. To create a project you simply: mix new thing ... where thing is the name of your project. Your project can be a big application or a small component. Phoenix builds on this and has a task of its very own: mix phoenix.new redfour Doing this will give me a phoenix site: mix phoenix.new redfour * creating redfour/config/config.exs * creating redfour/config/dev.exs * creating redfour/config/prod.exs * creating redfour/config/prod.secret.exs * creating redfour/config/test.exs * creating redfour/lib/redfour.ex * creating redfour/lib/redfour/endpoint.ex * creating redfour/test/views/error_view_test.exs * creating redfour/test/support/conn_case.ex * creating redfour/test/support/channel_case.ex * creating redfour/test/test_helper.exs * creating redfour/web/channels/user_socket.ex * creating redfour/web/router.ex * creating redfour/web/views/error_view.ex * creating redfour/web/web.ex * creating redfour/mix.exs * creating redfour/README.md * creating redfour/lib/redfour/repo.ex * creating redfour/test/support/model_case.ex * creating redfour/priv/repo/seeds.exs * creating redfour/.gitignore * creating redfour/brunch-config.js * creating redfour/package.json * creating redfour/web/static/css/app.css * creating redfour/web/static/js/app.js * creating redfour/web/static/js/socket.js * creating redfour/web/static/assets/robots.txt * creating redfour/web/static/assets/images/phoenix.png * creating redfour/web/static/assets/favicon.ico * creating redfour/test/controllers/page_controller_test.exs * creating redfour/test/views/layout_view_test.exs * creating redfour/test/views/page_view_test.exs * creating redfour/web/controllers/page_controller.ex * creating redfour/web/templates/layout/app.html.eex * creating redfour/web/templates/page/index.html.eex * creating redfour/web/views/layout_view.ex * creating redfour/web/views/page_view.ex Fetch and install dependencies? [Yn] This task created the skeleton of my web app, including: Tests for controllers, views and layouts which I don't want (the controller test fails when you change the index page template's text. That kind of thing drive me NUTS) A complete Brunch setup for handling my assets for me which is nice An Ecto Repository for data access A web directory, which is where models, controllers, views and channels go A smattering of other things I want about 60% of this. Forgive me - the first bits of this post sound kind of negative; they're not meant to be. I've learned the hard way over the past 10 years (or so) what can happen when you give control of the development process over to a framework and its conventions. These conventions are designed to get you off the ground fast but can (and have) make future maintenance a bit of a problem (see a few paragraphs above RE rewrites). What I want is this: functionality broken down into discrete components and processes that can be tested individually, that are isolated, and do not rely on a global framework to feel and embrace the love of OTP (Elixir/Erlang's underlying framework) at every level to not grow my web application into a monster Unfortunately the Phoenix guides walk you through a process of adding to the monolith - dropping a model and migration here, a controller there, adding a view and a template. There is so much more that's possible. Let's turn this in a positive direction. Creating Our App Directory Our application will be comprised of many parts; one of which is our web site. For this, Elixir's umbrella project structure will be perfect. It creates a structure for you that all your "sub applications" can live in - sharing dependencies while staying relatively independent: mix new redfour --umbrella * creating .gitignore * creating README.md * creating mix.exs * creating apps * creating config * creating config/config.exs Your umbrella project was created successfully. Inside your project, you will find an apps/ directory where you can create and host many apps: cd redfour cd apps mix new my_app Commands like "mix compile" and "mix test" when executed in the umbrella project root will automatically run for each application in the apps/ directory. Perfect. These little "sub applications" are loosely associated in that they share a dependencies folder (not the dependencies themselves - just the folder on disk), you can run your entire project's tests from the root, and you can reference each application using some syntactic sugar. Let's drop into the apps folder and install Phoenix. This time I'm going to strip it down to the bare essentials - I don't want to use an ORM-y database tool (Ecto) and I don't want to use Brunch; I have a way I like to manage/compress assets: cd redfour/apps mix phoenix.new web --no-brunch --no-ecto --module Redfour.Web ... Have a look in your /web directory. It's so trim and clean! Now have a look in the /deps directory in your root: ls -la ../deps drwxr-xr-x 11 rob staff 374 Feb 10 16:25 . drwxr-xr-x 9 rob staff 306 Feb 10 16:25 .. drwxr-xr-x 11 rob staff 374 Feb 10 16:25 cowboy drwxr-xr-x 10 rob staff 340 Feb 10 16:25 cowlib drwxr-xr-x 12 rob staff 408 Feb 10 16:25 fs drwxr-xr-x 14 rob staff 476 Feb 10 16:25 phoenix drwxr-xr-x 13 rob staff 442 Feb 10 16:25 phoenix_html drwxr-xr-x 9 rob staff 306 Feb 10 16:25 phoenix_live_reload drwxr-xr-x 9 rob staff 306 Feb 10 16:25 plug drwxr-xr-x 10 rob staff 340 Feb 10 16:25 poison drwxr-xr-x 9 rob staff 306 Feb 10 16:25 ranch A central place with all the dependencies for your project so you don't have bloat and dependency repetition. Ahhhhhhhhh. This is lovely. A stripped-down version of this radical framework that I can tweak as I need to. I like Controllers, I like the idea of Views (which I'll talk about more later on) and the routes/templating are great. There is so much goodness here, which I really feel is clouded a bit by the way it resembles Rails. Next Up: Our First Process We have our skeleton and we're ready to rock. As opposed to deciding up-front what data access I want, logging, infrastructure and everything else, I'm going to lean on OTP and think about things in terms of processes and (to a degree) monitored services. How would you do this for a commerce app? Let's see if our opinions match up for the next post. Feel free to leave a comment below. [Less]