Should i switch to Arduino Mega ?

I’m facing some troubles related with something i did not anticipate:

Program Size !

Arduino Uno Flash Memory is smaaalll !

As you may know, the ATmega328p use a “Harvard architecture”. This mean that program memory is separated from data memory.

Concerning the data memory, i already knew that the 2k bytes available on the Arduino Uno platform were not going to be enough for this project. That’s was why i anticipated that, and investigate a solution based upon the 23k256 chip (see this post).

But what i did not anticipated is that the program memory was also going to be too small. The Flash memory that contain program run by the Arduino Uno is 32k bytes:

  • 5k bytes are already used by the Arduino boot loader.
  • Linking and using the avr-libc + the Arduino Core Library also consume several kilo bytes
  • Linking and using the SPI + MIDI library + SPIRAM library + LiquidCrystal Library also consume several kilo bytes

The only thing, is that now, after writing some code for the MIDILooper itself, i only have less than 6k bytes available for all the rest of the code

I can’t be sure for  now, but it may not be enough to store the entire program…

There is no way to extend the Flash memory, so i may switch from Arduino Uno to Arduino Mega, which contain 512k bytes of program memory…


Extending the Arduino SRAM

What’s the problem with memory ?

That’s a good question… The ATmega328p offer 2k bytes of SRAM which is… not a lot…So knowing that, i tried to estimate how many MIDI command i can store inside this SRAM, and the result is problematic:

After reading the MIDI specification, i can estimate that most of the MIDI messages i’m going to store during the recording of a sequence are 3 bytes long: For exemple, when a note is played, the corresponding message contain the follwing bytes:

  • 1 byte of status: containing an ID representing the king of message, plus the MIDI channel ID
  • 2 bytes of data: containing the number the note (if it’s a C, a D or whatever) plus the velocity of the played note

2k bytes / 3 bytes = 682 messages

The point is that for each played not there is (at least) 2 MIDI messages: one when the note start (MIDI Note on) and one when the note end (MIDI Note off).

2k bytes / 3 bytes / 2 = 341 messages

But the estimation does not stop here… a MIDI message does not contain any timing informations (because they are supposed to be played in real time). If i want to playback those notes, i need to know WHEN to play them back, so i need to store an extra timing informations. To do that i’m going to follow what was already done in the MIDI file format.

A MIDI file contain a sequence of MIDI message and can be used to playback those messages/notes. In order to do that, extra data are added to each MIDI messages. These extra data are between 1 and 4 bytes long and store timing informations.

So if i assume this extra informations is always 4 bytes long, each MIDI message consume now: 3 bytes +4 bytes = 7 bytes. I will round it up to 8 bytes in case my application need to store some more extra data.

2k bytes / 8 Bytes / 2 = 128 messages

An other thing to take care of, is that among those 2k SRAM available, the Arduino System itself will already use some memory. And the application itself will also use memory. Here are differents estimations:

System take 128 bytes:
(2048 bytes – 128 bytes ) / 8 bytes / 2 =  120 messages

System take 256 bytes:
(2048 bytes – 256 bytes ) / 8 bytes / 2 =  112 messages

System take 512 bytes:
(2048 bytes – 512 bytes ) / 8 bytes / 2 =  96 messages

System take 1024 bytes:
(2048 bytes – 1024 bytes ) / 8 bytes / 2 =  64 messages

So i can reasonably think that i can store between 120 and 64 messages.

One last thing to note, is that when playing, not just MIDI note start and stop are send, other messages including “pitch variation“, or “note preassure” are also send. I can’t know in advance how many of this kind of messages are send because it of course depend on the way you are playing… But let’s say that for each note played, one of this commands is send this reduce the global estimation of total note that can be played to 32… This is not a lot and this is just an approximation…

So in order to avoid any memory related problems, additional memory is required !

Once again, thanks to the Arduino and it’s community, solutions exists:

The microchip 23k256 SPI SRAM:

The 23k256 is a small chip that provide 32k bytes of extra SRAM and can be easily connected to the Arduino:


Communication between this chip and the Arduino is done using SPI which is a serial communication protocol. (I will probably talk about the SPI protocol in an other post, but for now i will just talk about how to physicaly connect the chip and the Arduino ).

The tricky part here is that the 23k256 chip operate at 3.3V and the Arduino at 5V, so if i want to connect them safely, i must integrate some kind of voltage converter between the two devices…

The logic level voltage converter:

This kind of voltage converter can be done in very different ways, but, as i’m not an electronic specialist, i will use something that already exist: A 4 channel Bi-directional Logic Level Converter.

After reading the various specification for each devices, here is a first draft of how the wiring between the Arduino, the level converter and the 23k256, is going to be done…

I haven’t test it yet, but i think it should be ok 🙂

How everything should be wired (At least i think….):

23k256 wiring

Serial Communication: A “Don Quichotte” story !

Serial communication with Arduino :

The ATmega328p shipped with Arduino can do a lot of interesting things, including communication over several different protocols: SPI, I2E, and “classical” serial communication.

The serial communication is very useful as it allow us to send data to the host computer “easily” mainly for debugging purposes. Has i don’t know for the moment if a “real” debugger like GDB will operate withArduino, i will rely on simple “printf like” function to debug my code. That’s why Serial communication is pretty important.

Serial communication is provided by the ATmega328p used on Arduino Uno board. Arduino provide us facilities to use it, and among those facilities, there is the:

Serial over USB !!

This was the first point in this project where i get stuck for several hours trying to understand why things where not working as i except, i finally solve the problem, which one, like most of the time in programming, much more simpler then i thought !

In order for me to remember and for you to laugh at me here is the story :

Fighting against Windmill:

Because my development computer, a Raspberry pi, does not have any serial port, i need to use the usb port connected to the Arduino to communicate with it.

On the Arduino uno revision 3 this feature is provided by a small chip soldered on the board and connected to the ATmega328p. This chip, an ATmega8u2 (an other avr chip by atmel) is responsible of routing all the serial data going out of the main ATmega328p to the usb port of the arduino in the appropriate format.

When the arduino is connected to the Raspberry pi, a new tty is detected by the linux kernel:
It’s through this tty that you can read what the arduino send.

This sound pretty straight forward, but after uploading my first program to the Arduino, i did not receive any serial data on this tty…

At first i tough it was coming from a problem in my build options, or maybe from a wrong version of the Avr-Libc… so i spend several hours reading stuff about how the USB over serial was working until i realize something by looking at a typical sketch created using the Arduino IDE.

Roughly, when you write a sketch in Arduino IDE, you need to write only two function: Init() and Loop(). Those functions are then called by the code provided in the main.cpp file:

#include <Arduino.h>

int main(void)
#if defined(USBCON)
        for (;;) {
                if (serialEventRun) serialEventRun();
        return 0;

What is miss in my own program was the call to init(), which in defined in wiring.c. This function initialize some serial communication related stuff, and, as the comment in the code specifies:

“// this needs to be called before setup() or some functions won’t
// work there”

So after adding a call to init() in my own code, i was able to send and receive data over USB and start debugging my application ! Great !

Building Arduino Libraries

Among all the good things in the Arduino platform, is the huge amount of libraries. They either come from the standard Arduino distribution or from third party developer. And what’s even better is that they are all written in C++ ! Here is a list of some of the library i’m planning to use for this project:

  • Arduino distrib’s library :
    • Core Library:
      • Hardware serial communication library
    • Additional Library:
      • SPI library (SPI is a communication protocol used to connect chips or other devices to the Arduino)
  • Third party distrib:
    • SPI Ram library (i will probably need it to use the 23k256 chips from microchip that provide 256k bit of additional SRAM)
    • MIDI library used to receive and send MIDI data thru the MIDI shield

I will probably use other one, but for now these are the only one which sound really useful …

Compiling Arduino Core library:

So as Arduino is an OpenSource project, all the source code are freely available. You can get them here: In order to understand how to compile them, i have used mainly two sources of informations:

  • The Arduino web page explaining how to setup eclipse IDE to compile Arduino code (This article include a paragraph about how to compile Arduino Core Library)
  • The output of the Arduino IDE ( Check File->Preferences->Show verbose output during: compilation / upload )

After several tests, here are the GCC options i’m using for compiling the Core Library (and all the other one in fact !):

  • Compiling C Code:
    • -c : Do not invoke linker after compilation
    • -g : Generate debug informations
    • -Os: Optimize code to reduce it’s size
    • -Wall:  Enable all warning
    • -ffunction-sections : Place  functions item into its own section in the output file
    • -fdata-sections : Place  data items into its own section in the output file
    • -mmcu=atmega328p: Select the right type of AVR to use
    • -DF_CPU=16000000L: Specify the clock frequency of the AVH in hertz
    • -MMD: ???
    • -DUSB_VID=null: ???
    • -DUSB_PID=null: ???
    • -DARDUINO=105: Version of the Arduino libs
  • Compiling C++ Code:
    • All the previous one plus:
    • -fno-exceptions: Disable C++ exception support which are not supported by Arduino (And probably the underlying AVR hardware…)

Hate MakeFile, love CMake :

As i’m working under a Linux environment (Raspian on Raspberry Pi), i need some kind of MakeFile to drive the build process. I’m not a fan of this kind of file so i decided to go and use CMake to generate them for me! CMake is a great tool that allow you to write all your program/library specification using a simple language and which convert this specifications into a very complete Makefile or even a Project for the Microsoft Compiler !

Here is the Specification file i use to build the ArduinoCore Library:

Compiling others library:

Others libraries, including Arduino additional libraries and third party libraries are build the exact same way. So actually i’m building 4 library:

  • Arduino Core
  • Arduino SPI
  • Arduino SPI RAM
  • Arduino MIDI

Here is a link to the git hub project that host all of those library with the appropriate CMakeLists.txt (CMake specification file).

As always: Feel free to use it !

The programming Toolchain

Re-invent the wheel !

Most of the time, when i develop something for a personal project, i really like to start from scratch…It’s not about being a Linux or a command line extremist, it’s just that it allow me to really understand clearly what i’m doing.

This is one of the reason that drive my choice of not using the Arduino Integrated development environment. The other reason is that i want to program the entire project in C/C++ (which is my favorite language ) and that the Arduino IDE allow me to program in a “simplified” language which is not true C/C++.

Obviously, i made this choice after investigating what was feasible according to the basic tools available… Those research lead me to the following observation:

  • The Arduino IDE translate it’s “simplified” source code into true C/C++ during the compilation phase.
  • The Arduino IDE compilation process is based upon GNU avr-gcc which is an open source implementation of the GNU compiler collection for the AVR platform.
  • All the facility library used by Arduino are written in pure C/C++

So everything is already here to build a standard toolchain that can compile C/C++ code, use existing library (to do serial communication or MIDI processing…), and upload the program to the Arduino without a lot of pain..

Tools used:

So to summarize everything, here is what i will use to develop the software side of the MIDILooper:

  • Host computer: Raspberry PI.
  • Host OS: Raspian, a derived version of Debian compiled for the ARM architecture of the Raspberry.
  • Source Control: Git (using remote repository hosted on GitHub).
  • CMake for creating makefile allowing me to build the ArduinoCore library plus the MIDILooper program itself.
  • The GNU avr-gcc toolchain for compiling, assembling and linking my code.

So now, next step is to setup those tools and compile the Arduino Core Library !

Selected technical solution

Arduino to the rescue !!

Computer sciences and programming are great, but meeting and discussing with real people is even better !!

That’s a friend of mine that talk to me the first time about the Arduino solution.

Okay, i’m making some advertising, but you really should check out the website of his team Akwariom, they are creating amazing light control and audio technical solutions for live shows. YOU RULES DUDES !

So for those who don’t know about Arduino, in short, it’s an opensource hardware solution for creating small electronic device that can control pretty much everything ranging from stepper motor’s control, sensor data extraction to MIDI processing !

There are many variation of the Arduino platform, but the most used one is the Arduino Uno.

The Arduino Uno platform is mainly two things:

  • An hardware platform build around an Atmel AVR (ATmega328p) processor which is an 8 bit Risc microcontroller
  • A software solution that simplify the development process on this platform using a simplified programming language and several wrappers library.

As i want to develop this project in pure C/C++,  i’m going to use only the hardware platform, and develop my own tools for replacing the software solution, including the build toolchain and the wrappers library.

Arduino source code for the software solution and the wrapper library are available on GitHub:

Get rid of the Hardware side !

As i’m not an hardware specialist, i’m always disappointed when it come to solder resistors… But thanks to the Arduino platform, lots of company have developed hardware extension slot (called shields) for the Arduino platform.

Among them, there is one, SparksFun that have developed an extension shield that provide all the necessary MIDI circuitry needed for reading MIDI data and sending MIDI data:

It provide  3 MIDI connectors:

  • A MIDI IN which allow the Arduino to read MIDI data comming from an external device (in my case the Gaia SH01)
  • A MIDI OUT which allow the Arduino to send MIDI data to external device (again in my case, the GAIA SH01)
  • A MIDI THRU which is often present in MIDI powered device and which duplicate what come in the MIDI IN connector

Sounds like a great start no ? !

Nothing to solder and i like it !

Here is a picture of both the Arduino Uno and the MIDI Shield:


Genesis of the project

Let’s start !

So let’s start this blog with a first post introducing the project i’m planning to create, why i’m creating it and why it sound so cool to me !

MIDILooper ? So what’s that stuff ?

Good question ! Before that, let me quickly introduce myself… I’m a computer scientist since several years. Programming stuff, analyzing code, debugging programs, this is my things ! I spend ( … nearly 🙂 … )all my days reading and writing C/C++/python code for various stuff and in various domains (Video games, CG software, embedded software…)

Beside of that i’m also (at least trying to be…) a musician. I use the following instruments

  • A Seagull S6 acoustic guitar (recorded using a Audio Technica AT 2031)
  • A Yamaha P95 Digital piano.
  • A Novation Mininova synthesizer.
  • An Akai MPC500.

All of those instruments are mixed using a Behringer Xenyx 1002 Analog mixer.

The Piano, Guitar and synthesizer are then send back to the Roland Boss RC300 Loop-station. And the output of the loop-station is also mixed down to the Xenyx mixer.

The Boss RC300 Also act as a master controller: It send tempo ticks to the MPC500 and the Novation in order to keep them in sync with what i’m playing.

In short, i can record and loop 3 separates instruments tracks on the RC300 plus one extra track coming from the MPC500.

  • Here are some pictures of this setup:

IMG_20131118_203341 IMG_20131118_203414

  • And a nice graph showing how devices are connected:


So What ?

I’m very happy with the setup, but there one thing that annoy me: The 3 instruments tracks are audio track.

This is very good for piano, guitar or even voices, but for the Gaia synthesizer, it really reduce the expression potential. Let me explain:

This synthesizer have a huge amount of nobs and slider that can be tweak while playing in order to change the shape of the generated sound (for example change cutoff frequency of a low pass filter, or frequency of a LFO…)

The point is that when i record and then loop a musical phrase played on the GAIA, i cannot change the generated sound anymore. Thit is because the RC300 record the full audio data going out of the synth.
So as soon as the looped start, the sound is not generated anymore by the synth.

What i would like to do is the following:
Play a musical phrase on the GAIA, record it somewhere, loop that recorded phrase and WHILE the loop is playing, tweak nobs and sliders to change the sound played.

The theoretical solution:

Fortunately there is one theoretical solution, let me explain :
The Gaia SH01, like most of the actual synthesizer is made of 2 main “components”

  • A sound engine.
  • Controller.

The sound engine is what actually create the sound, you ask him: “Please play a C note for this amount of time, with this intensity and with this kind of sound” and he generate the sound you ear in real time. The type of generated sound depends on various settings controlled by several nobs and sliders. (The same nobs and sliders i want to tweak during a loop). Using those nobs and sliders, i can change the sound in real time, BUT, i have to tweak them WHILE i’m playing on the GAIA’s keyboard

The controller, to be simple, is the GAIA’s keyboard. When i play a note on the keyboard, a digital command is generated which contain all the informations about the played note. (But no sound, because sound is generated by the sound engine) This command is then send to the sound engine which play the specified note

Those commands are in fact MIDI commands.MIDI is a very popular standard allowing several musical devices to communicate with each others.

The Gaia have 2 MIDI plugs:

  • A MIDI OUT: All the MIDI commands generated by the keyboards are directly send to this output (in order to be send to the MIDI IN of an other instrument for example…)
  • A MIDI IN: The Gaia receive all the MIDI command send on this plug and play them directly using the sound engine.

So here is the goal of this project:
Create a small device, that can record a sequence of MIDI commands coming from an external device (the Gaia in my case), and that can send back this sequence to an other (or the same) device and loop that sequence over time.