Testing messages with Elixir

If you're coming to Elixir from another language, you might wonder how to assert that a process sent a message to another process.

The key is to remember that each test itself is also an Erlang process (and has its own pid).

We can send a message to the test process's mailbox, and then use `assert_receive` to assert we got the message.

For example:

test "something" do
  IO.puts "I am test, my pid is #{inspect self}"
  send(self, "hello test")

  assert_receive "hello test"
end

When you want to assert that a process sends a message, you just need to make sure the process is sending the messages back to the test's pid.

We do this by creating a stub that forwards messages.

Example

Say you have GenServer process that handles Payroll. Each time an employee is registered with the process, it sends a notification using GenEvent.
You want to assert that the event is sent.

This is what your Payroll server looks like:

defmodule Payroll do
  use GenServer

  # convenience function for starting up server, requires an event manager
  def start_link(manager) do
    # start GenServer with current module, and pass manager as state
    GenServer.start_link(__MODULE__, [%{events: manager}])
  end

  # initialize state
  def init([state]) do
    {:ok, state}
  end

  def handle_call({:add, employee}, _from, state) do
    # send out event
    GenEvent.notify(state.events, {:added, employee})

    # reply :ok
    {:reply, :ok, state}
  end

  # public API for adding employee
  def add_employee(pid, employee) do
    GenServer.call(pid, {:add, employee})
  end
end

It can be used like this:

# start a gen event server
{:ok, events} = GenEvent.start

# start the payroll service and pass the event manager's pid
{:ok, payroll} = Payroll.start_link(events)

# spawn another process that will print out events
spawn fn ->
  GenEvent.stream(events)
    |> Stream.each(&IO.inspect/1)
    |> Enum.to_list
end

# add an employee to the payroll service, which will result in notification being sent to previously spawned process
Payroll.add_employee(payroll, %{name: "Josh", salary: 1_000_000})

To test it, we'll need to create a stub GenEvent handler to forward messages back to the test:

# inside payroll_test.exs

# define a stub GenEvent handler
defmodule Forwarder do
  use GenEvent

  # handle all events, first parameter is the event, second if the state
  def handle_event(event, test_pid) do
    # send the event to `test_pid`
    send(test_pid, event)

    # respond `:ok` and keep the same state (the test's pid)
    {:ok, test_pid}
  end
end

# create all the processes we need in the setup
# ExUnit automatically terminates child processes after each test completes
setup do
  # create a GenEvent process
  {:ok, events} = GenEvent.start

  # start the payroll process, passing the event process's pid
  {:ok, payroll} = Payroll.start_link(events)

  # add the Forwarder as a handler, pass the current pid (self) as the state
  GenEvent.add_handler(events, Forwarder, self)

  # return test state
  {:ok, %{events: events, payroll: payroll}}
end

test "adding employee, sends notification", state do
  Payroll.add_employee(state.payroll, %{name: "Josh", salary: 1_000_000})

  assert_receive {:added, %{name: "Josh", salary: 1_000_000}}
end

And that is all folks..

Happy Elixiring!

How to Start

"I bought a book about procrastination several years ago. Haven't read it yet. I will get to it eventually."

Just kidding, I actually did read it ;)

Turns out procrastination is directly correlated to perfectionism. When perfectionism increases, procrastination increases (and vice versa).

So when you find yourself avoiding some task, it's very likely you've built up the task into something so big, so gargantuan, that starting has become daunting.

Take for example, the relatively basic task of baking cookies (I'm kind of hungry).

There are two ways to do this:

The Easy Way

Buy a bunch of ingredients (flour, sugar, chocolate chips, eggs, etc..) mix 'em up, throw 'em in a baking tray. BOOM, done!

Becoming a Perfectionist

After doing that for a while, you've now graduated to the "cookie expert" level, so you up your game. Instead of buying chocolate chips, you now buy bars of chocolate and chop them yourself. Heck, maybe you drive out to a farm and get some farm fresh eggs, maybe even freshly grind some rock sugar.

The results are phenomenal, so you keep adding and complicating.

Until.........

The World's Most Difficult (and Expensive) Artisinal Hand-Crafted Small-batch Farm-to-table Cookie

Recipe:

• Get a hen-house with chickens in the backyard, for super fresh eggs.
• Buy a small farm to grow wheat and harvest it for flour.
• Buy a sugar plantation to grow and harvest your own sugar cane.
• Raise a cow for fresh milk.
• Harvest your own cacao beans and make your own chocolate.

Congratulations, baking cookies just became harder than climbing Mt. Kilimanjaro.

You can no longer complete this task in 10 minutes, you now need a metric crap-tonne (actual SI unit) of planning, pre-planning and organizing.

You've created something too big to start.

Breaking the Cycle

To break the cycle, you'll need to create an easier start condition. You can do this by SIMPLIFYING AND REDUCING the first (aka starter) task.

Recall that procrastination effects only the start condition, STARTING the task (not actually doing the task). Once you've started, tiny amounts of success and accomplishment will build up and melt away procrastination (because you're creating mini gratification cycles).

Examples

• Have trouble working 8 hours? work for one hour and see how that feels, odds are you'll have no trouble continuing because the start was the only real barrier.
• Want to paint the deck but never get around to it? Tell yourself you'll only paint the railing, chances are you'll be done in a jiffy.
• Wanna run a mile? put on your shoes and run for just 5 minutes, bet you wont stop there.
• Never get around to writing that blog post? Just write one paragraph, 30 minutes later you'll realize you finished an entire post (actually how I wrote this)

Conclusion

Starting is always the biggest barrier. When you feel stuck, simplify and reduce until you reach a starter task so small that your "engines" can turn on. With your engines rolling, you're well on your way.

8 Ways Software is Priced

Buying software is really hard.

When done right, it's a huge asset for your business. It becomes the backbone of your business and helps attract investment.

When done wrong, it can cost thousands in modifying and fixing problems and will result in hurting (and possibly losing) valuable customers.

In this blog post, I'm going to explain how software is priced. I will show you the the pros and cons of each option, and how to ask the right questions and avoid common pitfalls. If you follow these tips, your next software purchase should be a resounding success.

If you have any questions, feel free to contact me at joshnuss@gmail.com

So like, how much does software cost?

It depends :)

That's like asking "how much does a car cost?", "how much does property cost?".

It depends on the type of car, the location of the property, etc.

A better question to ask is "what kind of car would I need?", "what kind of property is right for me?". There is no one-price-fits-all.

Software is an investment

Some business people will measure things purely on cost. I call them the "bean counters".

Bean counters will hire someone because their hourly rate is 5% lower than the next guy. They always look for the cheapest option. Bean counters think "I can do it cheaper by hiring someone in Albania for $0.25/hour". They neglect to look at skill, experience or track record.

Don't be a bean counter!

Instead, ask yourself what value is this project going to bring my business.

For example, say you have two employees working full time creating spread sheets, say each employee costs your business $35K/year + $10K/benefits. That means it costs you $90K/year to create these spreadsheets. If you automated this process, you'd be saving your business ~$450K over the next 5 years. Plus you could move 2 excellent employees into a more productive position.

So now you know how much these spreadsheets are costing you, how much should you pay to automate it?

You should aim for at least a 5X to 10X return. That means you can pay between $45K to $90K to solve this problem.

You may still be tempted to pay a lot less. Avoid this temptation, it will end up costing you in the long term.

Be a Unicorn

Every business is made up of parts that are unique and other parts that are common.

Ideally, you should carefully invest in unique parts of your business and outsource the remaining common parts.

For example, E-Bay has a unique way of selling products, so they have custom software for selling products. But when it comes to payroll, taxes or other non-unique aspects, they'd do best to outsource these systems, since they aren't innovating in payroll or taxes.

Remember: Always maintain maximum control over your business's core competency!

Cost structure

Software purchases are commonly priced in 8 different ways: Free, One-Time, Subscription, Time & Material, Retainer, Fixed-Bid, Shares and Salary.

Here's a breakdown of each, with suggestions on when to use the particular option and when to avoid it.

Free

There is no such thing as a free lunch.

Sometimes, software is marketed as free to attract new customers who will later turn into paying customers (by buying customizations or add-ons).

There are 2 types of free:

Proprietary: A company owns the code you are using, should you ever decide to stop working with that company, you cant use the software any more. This can lead to "vendor lock-in" where it becomes costly to leave because you'd need to start from scratch.

Non-Proprietary: (aka Open Source) The software is free to use and modify. You are not beholden to anyone, but you most likely will still need someone skilful to maintain it - its not totally free.

Use: Open-Source software is a great, provided its for a non-core competency.
Avoid: Proprietary stuff is often a trap, you'll be locked in with that vendor and you'll always second-guess whether you should leave.

One-Time

A one-time price is common in many off-the-shelf commercial software packages. Think Microsoft Office, Quickbooks, etc. These softwarez are sold as is and provide little post-sale customization opportunities. So you get what you buy and that's it, that's all. If something doesn't work the way you'd like, there's little recourse - other than to buy a competing package.

These offerings tend to suffer from over-complexity, they sell based on features, and end up with so many features that you have more knobs to turn than a 747 cockpit. It will lead to difficulty setting up the system, in day-to-day usage, and will require lots of staff training.

Use: Great for low-cost non-core commercial software.
Avoid: When something customized will save more time. E.g. don't spend $1 million on SAP when you can build something much cheaper and more specific.

Subscription

The subscription model is used in many on-line service websites (Shopify, GoDaddy, etc..). They offer a very low initial commitment, but are almost always proprietary, so you're completely locked in. If you move to another service you will have to start from scratch. Many suffer from 747 cockpit syndrome (way too many knobs and dials to turn).

Use: Good option for starting out on a low budget, or when the service is not a core-competency of your business.
Avoid: When service provided is a core competency of your business.

Time & Material

Many consultants and freelancers will charge this way (by hour or week), but it has some drawbacks. It puts extra risk on the buyer, because projects can be open ended (no agreed upon finish line), the buyer assumes the risk if things take longer (often the case). It can also be hard to compare rates between different skill levels, a really good developer charging $150/hour could be more economical than a junior developer charging $20/hour, because the quality and productivity could be 10 times better than the junior developer. Also, there is nothing stopping a company from charging a hefty hourly rate and using junior developers.

Use: Good approach for support/maintenance or very short projects.
Avoid: Bad for bigger projects because you (buyer) assume all the risk.

Retainer

Sometimes the most valuable thing you can get is good advise. One minute of good advise can save thousands of dollars. So many companies opt to have technical talent on hand in an advisory role. These are generally paid by retainer. It differs from Time & Material because unused time is still billed.

Use: Great for paying for advise, every serious company should have advisers on retainer.
Avoid: Not good for paying for project work (will be too expensive). Agree to a fixed price instead.

Fixed Bid

The price is fixed to a set of features. Commonly used by freelances, consultants and agencies. Easier for the buyer to budget. It also reduces the risk to the buyer, because if something takes longer than expected the price stays the same.

The only danger to the buyer is when requirements change during the implementation. It's quite common to surface new ideas during the course of implementation. To mitigate this risk, it is recommended to split a large contract into multiple fixed bid contracts (phases). That way you can still pivot requirements before its too late and costly.

Use: Projects with clear expectations.
Avoid: When expectations aren't clear or will change drastically over the course of implementation.

Shares

Some developers accept shares or revenue sharing in place of payment. Common in startups. Often mixed with other payment method (i.e.. Hourly rate + Shares). Some developers avoid it because it adds to their risk. Companies use this to add a valuable partner and ensure the resource is invested in the success of the business (and not just the success of the project).

Use: Want to decrease initial spend and have a valuable partner.
Avoid: Want to control ownership of business.

Salary

When you have a large budget and in house talent (or the desire to build talent) your best and cheapest option is to build it in house. This will offer your business the most control. Just make sure you have the budget to do this. It can add up when you need many different skillsets to complete a project. If it's not enought work to require a full time developer, you're better off hiring a consultant, freelancer part time or outsourcing to an agency. Remember that employees cost more than just their salary, there's benefits, office space, etc..

Use: Large budget core-competency projects where skills and manpower are are available in house.
Avoid: Non core-competency or manpower not available in house.

Tip: Mix and Match

The best approach is to mix several of these options to get the best results. For example, you could do the initial work with several rounds of fixed-bid with an agency, and then hire a developer employee to maintain your system in-house.

Conclusion

My grandma had this saying "Too poor to be cheap". It means the best way to invest in one's self or business is to focus on quality. You can't afford to re-buy the same object constantly. Buy objects that last a long a time - don't shop at the dollar store. Dont skimp on yourself or your business. Forget ODesk, forget Fiverr, do it right and you'll thank yourself later!

Joshua Nussbaum is Technology Integration Consultant in Montreal, Canada. You can reach me at joshnuss@gmail.com


Next up: 6 types of software companies

Why Design is Important: A Developer's Perspective

Last April, I found this really amazing project on Github (made by xxuejie). The project fascinated me, but the website was a bit hard to use - I wanted people to know about his project and enjoy it like I had.

So I forked the project, added a stylesheet and huge mess of jQuery and deployed the newly modified version. (xxuejie loved it, but didn't have time to maintain it, so we kept the forked version)

Original version: http://qiezi.me/projects/mruby-web-irb/mruby.html
Modified version: http://joshnuss.github.io/mruby-web-irb/

A few months later the modified version landed on HackerNews. It was well received, got tons of visits and was tweeted all over the place. There were blog posts in Japanese & Chinese. Even folks at my supposed "arch enemy" Microsoft blogged about it.

Most people would call such an experience satisfying, but for some reason I wasn't sure :)

On one hand, it was a huge compliment to have a project so highly regarded by your peers, on the other hand it kind of miffed me...

How could a bunch of styling changes and user interface upgrades make such a difference? My other projects seemed more intricate & technical but were never received in that fashion.

So I chewed on it for a bit.

It turns out I made the classic techie mistake of thinking in purely technical terms. I had factored out the human element.

For us humans, vision is (arguably) our most important sense (highest bandwidth). When an object doesn't communicate a compelling visual element, it leaves an important channel to the brain unused and negatively impacts its usability potential.

Put another way, it doesn't matter how technically awesome something is - if it fails to register with a human brain, it will never be relevant. Some techies might consider that unfortunate, I don't (anymore), its just how the world works.

Without good design (and for that matter good content, marketing, information architecture & sales strategies) our software & hardware would live an inconsequential existence. Technology is just one piece of the puzzle.

FPGA Logic Cells

Lately I've seen lots of homebrew CPUs mentioned in blogs and videos, but not too much about homebrew FPGAs. Did a little digging (mostly on Wikipedia), and was able to breadboard a basic logic cell circuit. Here's how I did it.

A single logic cell. Using an Arduino to program the memory. Once the cell is programmed, the programming cable can be disconnected and the cell retains the truth table in memory.

Background

A Field Programmable Gate Array is a matrix of reprogrammable logic blocks. Each logic block contains multiple logic cells.

Each cell can be programmed to function as any type of logic gate! i.e. AND, OR, XOR etc..

This matrix of blocks are then woven together in a mesh that allows blocks to interconnect with other blocks, and of course these connections can be reprogrammed any time.

Today, I will focus on the logic cell.

Logic Cell

It performs a single boolean operation based on a pre-programmed truth table. Unlike when building gates from CMOS or TTL logic chips directly, The truth table is reprogrammable.

For 2 input truth tables, there are 2^2=4 possible outcomes. Each of those outcomes is stored in a memory. We will use four D-Type Flip Flops (74HC174) to get the memory we need.

D-Type Flip Flop

The D-Type Flip Flop is a positive edge triggered flip flop. That means it will store the value of the data pin only when the clock pin goes high. After that, no matter what input it gets, the last value will be held, until it is cleared or a new value is clocked in. We can take 4 of these guys and tie the Clear and Clock pins together and voila - we have a 4 bit memory suitable for storing a truth table.

Lookup Table

For example, consider an XOR gate, the four possible outcomes are (0,1,1,0)

So now that we have that in our 4-bit memory, we need a way to select one of those four values depending on what the given 2 inputs are. For that we will use a 4-to-1 Multiplexer 74HC153 (also knows as decoder or selector). Or for any of you programming nerds out there, the hardware world's equivalent to a switch statement.

The 4-to-1 multiplexer has 4 possible outcomes labeled A-D and has 2 inputs called S0 and S1, the output is labeled Y. In pseudocode it would look something like this:

If S0=0 and S1=0 then Y=A
If S0=0 and S1=1 then Y=B
If S0=1 and S1=0 then Y=C
If S0=1 and S1=1 then Y=D

Logic cells can operate both in parallel or based on a clock. So another 1-bit memory is used to store whether the cell is clocked or not.

For now I will skip the clocking part, it is easy to add it. For now I will focus on the programmable lookup table.

Schematic

Breadboard

Example Breadboard Layout

Programmer Firmware

An Arduino can be used to program the 4 bit memory over a serial connection. Here is an example sketch:

#define A      2
#define B      3
#define C      4
#define D      5

#define CLOCK 10
#define CLEAR  9
#define HOLD   8

void truth_table(bool a, bool b, bool c, bool d) {
  // print truth table
  Serial.println(a ? "A=0,B=0,Y=1" : "A=0,B=0,Y=0");
  Serial.println(b ? "A=1,B=0,Y=1" : "A=1,B=0,Y=0");
  Serial.println(c ? "A=0,B=1,Y=1" : "A=0,B=1,Y=0");
  Serial.println(d ? "A=1,B=1,Y=1" : "A=1,B=1,Y=0");
  Serial.println("");

  // clear flip flops  
  digitalWrite(CLEAR, LOW);
  digitalWrite(CLEAR, HIGH);
  
  // flip bits where neccessary
  digitalWrite(A, a ? HIGH : LOW);
  digitalWrite(B, b ? HIGH : LOW);
  digitalWrite(C, c ? HIGH : LOW);
  digitalWrite(D, d ? HIGH : LOW);

  // toggle clock
  digitalWrite(CLOCK, HIGH);
  digitalWrite(CLOCK, LOW);
}

void setup() {
  pinMode(A,     OUTPUT);
  pinMode(B,     OUTPUT);
  pinMode(C,     OUTPUT);
  pinMode(D,     OUTPUT);
  pinMode(CLOCK, OUTPUT);
  pinMode(CLEAR, OUTPUT);
  pinMode(HOLD,  OUTPUT);

  // initialize clock pin
  digitalWrite(CLOCK, LOW);
  
  // initilize clear pin
  digitalWrite(CLEAR, HIGH);

  // initialize truth table
  truth_table(false, false, false, false);

  Serial.begin(9600);
}

char values[4] = {'F','F','F','F'};
char current;
int index = 0;  

void loop() {
  
  while (Serial.available()) {
    current = Serial.read();
    values[index] = current;
    index++;
   
    if (index == 4) {      
      truth_table(values[0] == 'T', 
                  values[1] == 'T', 
                  values[2] == 'T', 
                  values[3] == 'T');
      index = 0;
    }
  }
  
}

Using the Programmer

Open the Serial Monitor @ 9600, no line endings
To send a truth table to the device, you will need to type 4 letters, either capital "T" or capital "F".

Examples:

For an AND gate type FFFT the LED will only light up when both buttons are pressed
For a NAND gate type TTTF the LED will light up except when both buttons are pressed
For an XOR gate type FTTF the LED will only light up when only one of the buttons is pressed

Now on to logic blocks and routing :)

Find last updated date for ActiveRecord model

Often a page needs to show when content was last updated, its easy to do when showing a particular record, but what about the last updated date for a list of records?

This is a little snippet I use:

date_of_last_fee_change = Fee.last_updated

To make it happen for all models:

ActiveRecord::Base.class_eval do
  def self.last_updated
    last = first(:order => 'updated_at DESC')
    last.try(:updated_at) 
  end 
end

Easily manage multiple apps

Sometimes you want to run a command on each of your rails apps. i.e. git pull or rake db:migrate

A simple ruby script can handle that:

each-app.rb:

#!/usr/bin/ruby

PATH = '/mnt/apps'

Dir.glob("#{PATH}/*").each do |dir|
  Dir.chdir(dir)
  puts "(in #{dir})"
  ARGV.each { |cmd| system(cmd) }
end

This script iterates each folder under "/mnt/apps" directory, goes into that directory and executes each command.

you can run it like so:

./each-app.rb 'git pull' 'rake db:migrate'

just dont forget to

chmod +x each-app.rb

happy hacking

Why dont Array.first() and Array.last() accept blocks?

It would be nice if Array.first and Array.last took block parameters. That way you could get the first or last item in the array based on a block condition. Example:

>> [1,2,3].first {|x| x > 1 }
=> 2

As most things in ruby, its super easy to implement:

class Array
 def first
   each do |x|
     return x if !block_given? || yield(x)
   end
 end
 def last
   reverse_each do |x|
     return x if !block_given? || yield(x)
   end
 end
end

a = [1,2,3]
puts a.first            # => 1
puts a.first {|x| x>1 } # => 2
puts a.last             # => 3
puts a.last  {|x| x<2 }  # => 1

Heres how to implement this with the MRI interpreter. (Not as beautiful)

static VALUE
rb_ary_first(argc, argv, ary) 
    int argc;
    VALUE *argv;
    VALUE ary; 
{
    if (argc == 0) { 
        if (rb_block_given_p()) {
                long i;
                RETURN_ENUMERATOR(ary, 0, 0);
                for (i=0; ilen; i++) {
                        if (RTEST(rb_yield(RARRAY(ary)->ptr[i]))) {
                                return RARRAY(ary)->ptr[i];
                        }
                }
                return Qnil;
        } else {
                if (RARRAY(ary)->len == 0) return Qnil;
                return RARRAY(ary)->ptr[0];
        }
    }    
    else {
        return ary_shared_first(argc, argv, ary, Qfalse);
    }
}

Quicksort

Array#sort works great, but its still fun to write a quick sort in ruby!

class Array
 def quicksort
   return self if length <= 1

   pivot = shift
   less, greater = partition {|x| x <= pivot}

   less.quicksort + [pivot] + greater.quicksort
 end
end

puts [8,99,4,1000,1,2,3,100,5,6].quicksort.join(',')

Paged Enumerable

Scraping some pages with WWW::Mechanize and finding myself needing to download multiple pages. So I wrote a paged enumerable module. Paged enumerable does everything Enumerable does, except instead of implementing a meaningful each you implement a meaningful each_page which yields an array or items per page.

to use the paged enumerable, you would write a class like this:

class MultiplePageSearch
  include PagedEnumerable

  def each_page
    10.times { |page| yield download_page(page) } # simulate 10 pages
  end 

private
  def download_page(page)
    puts "downloading page #{page}..."
    sleep 1 # simulate slooow operation
    start = page*10

    start...(start+10)
  end 
end

paged = MultiplePageSearch.new
puts paged.any? {|x| x > 50} # will only hit 5 pages
paged.each {|x| puts x} # will process all pages

The implementation of PagedEnumerable is quite simple

module PagedEnumerable  
  def self.included(obj)
    obj.send :include, Enumerable
  end 

  def each(&blk)
    each_page { |page| page.each(&blk) }
  end
end

and in most cases you would cache the pages for better performance on a second pass.