The trouble with board files:
The old 3.2 Kernel worked pretty well, so why was a move made to 3.8, which causes so much grief to the beaglebone old timers?
The major problem with the 3.2 kernel is that uses a board-file, namely,
arch/arm/mach-omap2/board-am335xevm.c
It's pretty big (at 4K+ lines), and quite difficult to modify.
The boardfile also supports, in addition to the beaglebone white, the
am335x-evm and all the capes. Every new cape that is to be supported has
to modify this file. Additionally you need to create the platform
devices that it contains and register them. You have to make sure that
you don't break any of the other boards (or capes) supported by the
platform file.
This is a job for professional kernel developers, but even kernel
developers have their limits as evidenced by the famous Linus rant that
prompted the great cosmic shift to Device Tree:
Famous Linux rant about ARM churn
What has been decreed is simple:
“No more new ARM board files”
So all new boards must support booting using Device Tree. The
board file used in the new kernels is just
arch/arm/mach-omap2/board-generic.c and supports booting kernels using
omap2, omap3, omap4, omap5 and the am33xx family of the beaglebone.
This presented a difficult decision for the developers working on the beaglebone black Linux support:
- Ignore the new mainline dictums, carry on using 3.2 and make the
jump to mainline sometime in the future. That would require more
developer resources, when the time for mainline porting comes. On top of
that most kernel developers don't bother with older vendor kernels, and
any bug fix or new functionality has to be painfully back-ported from
mainline.
- Bite the bullet, move to 3.8 using DT, and future proof the
beaglebone by submitting everything for inclusion to mainline as soon as
possible.
The trouble with Device Tree:
What is Device Tree (DT) and why is it causing so many problems for new developers?
Device Tree Central
"The Device Tree is a data structure for describing hardware.
Rather than hard coding every detail of a device into an operating
system, many aspect of the hardware can be described in a data structure
that is passed to the operating system at boot time."
(Please take note of the 'boot time' reference, we'll come back to it later).
The pinmux subsystem and clocksource/clockevent frameworks are
two examples of consolidated frameworks that have evolved from the ARM
move to DT. Previously, each ARM "machine", such as mach-omap2/*,
implemented their SoC-specific pinmux and timer/clock drivers within
their machine directory. In converted ARM machines, these drivers now
live in the appropriate drivers/* directory and share common code sanely
with other similar drivers.
As it turns out DT isn't that new, it's a couple of decades old
and has been used by Open Firmware based systems like Sun workstations
and the original PowerPC based Apple Macs.
For developers the biggest change is that the kernel board file is now gone.
Simply put, every board has to be described in a DTS (foo.dts)
file that is compiled via the DTC compiler to a binary format DTB
(flattened device tree format) which gets parsed on boot by the kernel
and the devices are created.
Therein lies the biggest change; there are no more platform
device data for the device containing it's configuration. On a board
file based platform what you did was:
In the foo_platform_data.h file:
struct foo_platform_data {
u32 bar;
};
In the board file:
struct foo_platform_data foo_pdata {
.bar = 5,
};
struct platform_device foo_dev {
.name = "foo",
.id = -1,
.dev.platform_data = &foo_pdata,
};
And in the board setup function
platform_device_register(&foo_dev);
The driver gets access to the platform data in the probe function.
static int foo_probe(struct platform_device *pdev)
{
struct foo_platform_data *pdata;
pdata = pdev->dev.platform_data;
if (pdata == NULL) /* ERROR */
...
/* access pdata->bar for configuration */
...
}
static struct platform_driver foo_driver = {
.probe = foo_probe,
....
.driver = {
.name = "foo",
},
...
};
module_platform_driver(&foo_driver);
This method no longer works; in a DT based system what you have to do
come up with device driver bindings, which contain the configuration
the driver requires.
You must add a device node in board.dts under the on-chip-peripherals(OCP) device node:
foo {
compatible = "corp,foo";
bar = <5>;
};
No change in the board file (generic anyway) needs to be made, but
the device driver must be updated to support the DT method, with
something similar to the following:
static struct foo_platform_data *
foo_parse_dt(struct platform_device *pdev)
{
struct device_node *node = pdev->dev.of_node;
struct foo_platform_data *pdata;
pdata = devm_kzalloc(&pdev->dev, sizeof(*pdata), GFP_KERNEL);
if (pdata == NULL)
return NULL; /* out of memory */
/* no such property */
if (of_property_read_u32(node, "bar", &pdata->bar) != 0)
return NULL;
/* pdata->bar is filled in with 5 */
return pdata;
}
static int foo_probe(struct platform_device *pdev)
{
struct foo_platform_data *pdata;
if (pdev->dev.of_node != NULL)
pdata = foo_parse_dt(pdev);
else
pdata = pdev->dev.platform_data;
if (pdata == NULL) /* error */
...
}
static const struct of_device_id of_foo_match[] = {
{ .compatible = "corp,foo", },
{ },
};
static struct platform_driver foo_driver = {
.probe = foo_probe,
....
.driver = {
.name = "foo",
.of_match_table = of_match_ptr(of_foo_match),
},
...
};
module_platform_driver(&foo_driver);
The driver described above can support both the platform data model
and the new DT method. Using a purely DT driver model can result in more
flexibility, but we won't deal with it now. Frankly, the biggest change
is that now a user (by compiling the DTS) can now configure his
platform without having to recompile the kernel, which is a pretty big
usability gain. We use this in the new 3.8 releases to boot both
versions of the beaglebone using the same kernel, and only changing the
DTB we boot with.
For a developer, the biggest change of using DT, is that it is
purely data driven, and that custom platform drivers are frowned upon,
opting instead to using generic frameworks, one of which is the pinmux
framework.
In summary, the core of the change to DT is that logic that was
previously accomplished in board files is moved into a mixture of
generic frameworks, abstracted devices drivers, and data to drive that
software. DT imposes structure upon the embedded developer and sometimes
there is considerable pushback.
Device Tree's data driven model
The data driven model of device tree causes the most pain to the
embedded developer steeped in the board file/platform method. Let's
examine a concrete example, that deals with the simple subject of a
driver/board pinmuxing.
Pinmuxing refers to the method of modern SoC multiplexing
multiple peripheral functions to a rather limited set of physical
pins/balls of the SoC package. The am335x is a prime example this,
having many pins with up to 8 possible peripheral functions.
For a peripheral driver to work the correct muxing configuration
must be applied to the affected pins; since they are so many, and
configuration is complex, a special omap specific driver has been
written and has been used on the 3.2 kernel provided by TI.
Typically in the board setup file, the mux configuration for the peripheral is applied before registering the platform device.
For example the pin mux for the a gpio configured as a led would be of the form
omap_mux_init_signal("gpmc_a2.gpio1_18",
OMAP_MUX_MODE7 | AM33XX_PIN_OUTPUT);
The omap mux driver also provides a user-space interface for changing
the mux settings. The problem with pinmuxing is that it is potentially
dangerous if you don't know exactly what kind of hardware is connected
to the SoC pins, so in the mainline pinmuxing is performed via the
drivers on probe time.
For the sake of making the example more concrete let's have a
gpio also controlling power to the device. The device's driver is only
supposed to apply the mux and turn on the power to the device by setting
a gpio to high (please bear in mind this is not a real-world example,
but explains the issues).
Let's examine the platform data example with a call back method.
In the foo_platform_data.h file:
struct foo_platform_data {
u32 bar;
void (*get_ready)(struct platform_device *pdev);
};
In the board file:
/* magic GPIO number of FOO’s power pin */
#define FOO_POWER 11234
static void foo_get_ready(struct platform_device *pdev)
{
int ret;
omap_mux_init_signal("gpmc_a2.gpio1_18",
OMAP_MUX_MODE7 | AM33XX_PIN_OUTPUT);
ret = gpio_request(FOO_POWER, "foo-power");
if (ret >= 0)
gpio_direction_output(FOO_POWER, 1);
}
struct foo_platform_data foo_pdata {
.bar = 5,
.get_ready = foo_get_ready,
};
The rest of the board file has no other changes.
The driver then can call the get_ready method via the callback function pointer.
static int foo_probe(struct platform_device *pdev)
{
struct foo_platform_data *pdata;
pdata = pdev->dev.platform_data;
if (pdata == NULL) /* ERROR */
...
/* access pdata->bar for configuration */
...
/* get foo ready */
(*pdata->get_ready)(pdev);
}
Callback functions are impossible in a DT platform. Callbacks break
the data only model of DT. What must be done is for the driver to be
converted to using the generic gpio and pinmux bindings. On am33xx the
generic pinctrl-single driver is used, and to be honest, is not as
polished as the old omap mux driver. Nevertheless it is usable and used
on many other boards as well, since it is widely supported in the ARM
community. Again generic frameworks win over custom solutions.
The boot.dts file is updated with the pinmux nodes and the gpio bank nodes.
/* point to the to the conf_* module registers */
am33xx_pinmux: pinmux@44e10800 {
compatible = "pinctrl-single";
reg = <0x44e10800 0x0238="">;
#address-cells = <1>;
#size-cells = <0>;
pinctrl-single,register-width = <32>;
pinctrl-single,function-mask = <0x7f>;
foo_pins: foo_pins {
pinctrl-single,pins = <
/* need to look into the TRM and find those values */
/* conf_gpmc_a2 module config reg offset and value */
0x0A8 0x1f
>;
};
};
gpio2: gpio@4804c000 {
compatible = "ti,omap4-gpio";
ti,hwmods = "gpio2";
gpio-controller;
#gpio-cells = <2>;
interrupt-controller;
#interrupt-cells = <1>;
reg = <0x4804c000 0x1000="">;
interrupts = <98>;
};
foo {
compatible = "corp,foo";
bar = <5>;
# the power control gpio
power-gpio = <&gpio2 18 0x0>;
# the muxing
pinctrl-names = "default";
pinctrl-0 = <&foo_pins>;
};
The new nodes are the am33xx_pinmux node, and the gpio2 node. Note the
added node in am33xx_pinmux that contains the pinmux config for the foo
driver. Please note due to the OMAP's numbering of peripherals starting
with 1 every numbered peripheral is +1 from what the AM3359 TRM states.
There are patches against mainline that fix that, but not on 3.8 yet.
The foo driver only need minor changes, first the pinctrl consumer interface include header must be included.
#include < linux/pinctrl/consumer.h >
Then on the probe function we need to apply the pinctrl
configuration, while on the parse_dt function we need to set the gpio to
the given value.
static struct foo_platform_data *
foo_parse_dt(struct platform_device *pdev)
{
struct device_node *node = pdev->dev.of_node;
struct foo_platform_data *pdata;
enum of_gpio_flags ofgpioflags;
int ret, gpio;
pdata = devm_kzalloc(&pdev->dev, sizeof(*pdata), GFP_KERNEL);
if (pdata == NULL)
return NULL; /* out of memory */
/* no such property */
if (of_property_read_u32(node, "bar", &pdata->bar) != 0)
return NULL;
/* now get the power gpio and set it to one */
gpio = of_get_named_gpio_flags(pdev->dev.of_node, "power-gpio",
0, &ofgpioflags);
if (!IS_ERR_VALUE(gpio)) {
gpioflags = GPIOF_DIR_OUT;
if (ofgpioflags & OF_GPIO_ACTIVE_LOW)
gpioflags |= GPIOF_INIT_LOW;
else
gpioflags |= GPIOF_INIT_HIGH;
ret = devm_gpio_request_one(&pdev->dev, gpio,
gpioflags, "foo-power");
}
/* pdata->bar is filled in with 5 */
return pdata;
}
static int foo_probe(struct platform_device *pdev)
{
struct pinctrl *pinctrl;
...
/* apply the pinmux configuration */
pinctrl = devm_pinctrl_get_select_default(&pdev->dev);
if (IS_ERR(pinctrl))
dev_warn(pdev->dev, "unable to select pin group\n");
if (pdev->dev.of_node != NULL)
pdata = foo_parse_dt(pdev);
else
pdata = pdev->dev.platform_data;
….
}
Cape Manager requirements
Going over the device tree definition we see that the data structure
is referred as parsed at boot-time. Beaglebone capes are not static; a
different cape set might be connected each time the board boots, and the
only way to find out what kind of cape is connected is to read the
serial EEPROM present on each cape which contains identification
information.
Beaglebone capes are mostly compatible between White and Black,
but due to the presence of the eMMC flash and the HDMI framer, some
special attention must be paid for capes that use the same resources.
The information stored on the EEPROM of each cape are a user
readable name, serial number, part number, revision information and
others. The only information that the cape manager uses to work are the
part-number and the revision.
A cape-manager should perform the following:
- Scan the I2C bus for EEPROMs that correspond to the address defined
in the beaglebone SRM. Those addresses are 0x54-0x57 on I2C1 bus, hence
the 4 capes limitation.
- Read the EEPROM contents and parse them according to the spec. Retrieve the part-number and revision information of that cape.
- Match the PART-NUMBER:REVISION tuple to a configuration that would
result in the system state being that as if the cape was directly
attached to the base board, and perform any actions that result to it.
Additionally we need extra facilities that are useful for people developing capes:
- An override facility so that prototyping capes with no EEPROM can be supported.
- An option to load capes at runtime.
- An option to unload capes at runtime.
- An option to stop a cape from loading even if detected.
- To guard against resource conflicts. A cape that uses the same
resources as another loaded earlier should not be permitted to load.
- To be able to forbid cape loading of capes that are not supported by a baseboard revision.
- To be able to load 'virtual' capes that correspond to peripheral that are present on specific baseboard revisions.
- To support the notion of a cape priority, so that in case of a
conflict, the cape we define as higher priority gets loaded. This is
important to allow plug in capes to take preference over capes that are
part of the baseboard; i.e. when an LCD cape is attached it takes
preference over the onboard HDMI virtual cape.
- To not require source changes for any driver for a peripheral that
resides on cape or onboard , i.e. the drivers should not be modified for
use in a cape.
- No kernel source changes must be required for a new cape to be supported.
- To not require modification of the boot-loader, or of the base DTS.
Device Tree Overlays
The method used to dynamically load the cape definition to the
running kernel is called a Device Tree Overlay. What a device tree
overlay does is apply changes to the kernel's internal DT representation
and trigger the changes that this action carries. For example adding a
new enabled device node should result in a new device being created,
while removing a node removes the device and so on.
The previous foo example, when reworked for usage with DT overlays is as follows:
The base.dts file has an ocp label of the on-board peripherals for the SoC.
/ {
/* On chip peripherals */
ocp: ocp {
/* peripherals that are always instantiated */
peripheral0 { ... };
}
};
The overlay when loaded must result in the same device tree as in the previous example; the overlay must be the following:
/plugin/; /* allow undefined label references and record them */
/ {
.... /*various properties for loader use; i.e. part id etc. */
fragment@0 {
target = <&am33xx_pinmux>;
__overlay__ {
foo_pins: foo_pins {
pinctrl-single,pins = <
/* look into the TRM for values */
/* conf_gpmc_a2 module reg offset/value */
0x0A8 0x1f
>;
};
};
};
fragment@1 {
target = <&ocp>;
__overlay__ {
/* bar peripheral */
foo {
compatible = "corp,foo";
bar = <5>;
# the power control gpio
power-gpio = <&gpio2 18 0x0>;
# the muxing
pinctrl-names = "default";
pinctrl-0 = <&foo_pins>;
};
};
};
};
The overlay consists of two fragments, one targeting the
am33xx_pinmux node, the other the ocp node. Note that there are no foo
driver changes, not any sort of platform board file hacking. When this
overlay is applied the foo device will be created, just as if it was
declared in the base.dts.
More information about the internal of overlays and the
resolution mechanism is located in Documentation/devicetree directory of
the kernel.
Cape Manager and Device Tree Overlays
Device Tree overlays is a general purpose mechanism, which is not
tied to any platform. For actual platforms like the beaglebone the
mechanism must be supplemented by platform specific logic.
The platform logic of the beaglebone cape manager deals with the following:
- Reading the EEPROMs of the capes for retrieval of the part numbers & revision.
- Allowing runtime addition/removal of cape fragments.
- Having options to control cape loading from the kernel command line
- Having options to force loading of capes from the base DTS
- Having mapping options of multiple cape part numbers & revisions to a single cape
- Resource conflict management
- Implement cape priorities so that in case of a conflict the winner can be selected
- Verification of the cape compatibility with a base board
- Various cape information display (i.e. serial# etc).
- Automatic loading of capes based on base board version (i.e. auto loading HDMI/eMMC capes)
Lets add the beaglebone specific properties to the foo DT overlay and complete the cape.
/plugin/; /* allow undefined label references and record them */
/ {
/* compatible with both the original and the beaglebone black */
compatible = "ti,beaglebone", "ti,beaglebone-black";
/* part-number */
part-number = "BB-FOO-01";
/* version */
version = "00A0";
exclusive-use =
"P8.43", /* Header.Pin# */
"gpio2_8"; /* the hardware IP we use */
/* the fragments are the same as above */
....
};
The compatible property lists the platforms that this cape supports.
If an attempt is made to load an unsupported cape by this platform the
attempt will fail.
Similarly, the part-number and version properties guard against an attempt of loading a bad cape.
The exclusive-use property is a pretty simple attempt at
implementing resource management. What it does, is allow capes to
declare free-form text resources; if another cape tries to load which
uses the same resource, it will be prevented of doing so. The convention
in the beaglebone cape format is that we reserve the header/pin and the
hardware IP block the cape wants to use, in that case pin#43 of the P8
connector, and the gpio2_8 gpio hardware block.
The name of the cape should be of the ${PART_NUMBER}:${REV}.dts
format and the compilation would be (on either the host or the
beaglebone):
$ dtc -O dtb -o BB-FOO-01-00A0.dtbo -b 0 -@ BB-FOO-01-00A0.dts
Note the required -@ option; it is not part of the upstream dtc, so
it's a DT overlay specific patch that enables it. The beaglebone kernel
has the patch already applied on it’s internal dtc compiler copy, as
well as the angstrom image, but you need the following patches for other
distro’s dtc packages.
Dynamic Overlays DTC patch
Putting the resulting BB-FOO-01-00A0.dtbo file in /lib/firmware we can enable it by
# echo BB-FOO-01 >/sys/devices/bone_capemgr*/slots
We can verify if it loaded with:
# cat /sys/devices/bone_capemgr*/slots
0: 54:PF---
1: 55:PF---
2: 56:PF---
3: 57:PF---
4: ff:P-O-L Bone-LT-eMMC-2G,00A0,Texas Instrument,BB-BONE-EMMC-2G
5: ff:P-O-L Bone-Black-HDMI,00A0,Texas Instrument,BB-BONELT-HDMI
6: ff:P-O-L Override Board Name,00A0,Override Manuf,BB-FOO-01
Slot #6 is the one we've loaded.
The cape can be removed by:
# echo -6 >/sys/devices/bone_capemgr*/slots
Cape-manager also supports a method of enabling (or disabling) a cape by a kernel command line option.
capemgr.extra-override=BB-FOO-01
Will automatically load the cape on boot, while
capemgr.disable_partno=BB-FOO-01
will disable the cape (if it's attempted to load automatically).
Porting to the new DT kernel FAQ
Q: I have a custom kernel fork, based on a modified board file. Nothing works.
A: The biggest difference of-course is that there's no board
file, so there are no platform data. You have to have device drivers
with DT bindings. You can get by temporarily by creating a helper DT
enabled platform device and allocate/fill the platform data from the
probe method and register the platform device. But this is only a
temporary fix, and DT bindings should be introduced at some point in the
driver.
Q: The new pinmuxing method sucks; why can't I pinmux by the old method at runtime?
A: The old pinmux method was TI specific and although it made it
in the mainline, it has no DT bindings and is duplicating functionality
of the general pinmux framework. The new method is portable, but it's
reliance on raw numbers is a pain. However it does offer safety, since
if a device obtains the pin resources there's no way for them to be
modified by some outside factor. To help in the transition a full set of
peripheral drivers and their
pinmux settings will be introduced shortly.
Additionally there is a pinmux helper driver that allows runtime
switching between different per-state pinmux-settings. For example:
bar_pinmux_helper {
compatible = "bone-pinmux-helper";
status = "okay";
pinctrl-names = "input", "output";
pinctrl-0 = <&bar_input_gpio_pins>;
pinctrl-1 = <&bar_output_gpio_pins>;
};
This device node creates a pinmux helper device which defines two
pinmux states, named input and output. You can switch between them
simply by
# echo input >/sys/devices/ocp*/bar_pinmux_helper*/state
# echo output >/sys/devices/ocp*/bar_pinmux_helper*/state
Note that there is a very slim chance for this driver to be admitted
in mainline, but you can use it if it helps you port to the new kernel.
Q: How do I support any kind GPIO/I2C/SPI etc. device. It is completely different.
A: It is. Examples will be provided on how to do most of the
standard peripheral configurations. What you get for the change is the
ability to create any kind of device without having to recompile the
kernel.
Q: My LCD/GPMC/$RANDOM based device doesn't work on the black.
A: The eMMC and HDMI parts of the Black are using the same pins.
If you define a cape with the proper resource definitions, the user cape
will take precedence. Hardware restrictions might apply (i.e. pin
loading issues etc).
Q: Why can't you carry on using the board file method, seems to work just fine.
A: The writing was on the wall; no way we could ever get the
beaglebone in the mainline using it. On top of that any new cape that is
produced, either by TI/CCO or the community had to patch the board file
to make it work. While this is doable when you have a couple of capes,
when you expect dozens of capes, most of them produced by community
members, having to expect them to hack the board file, is completely out
of the question.
Q: The pinmux values are really hard to figure out from the
manuals. One has to hunt in the TRM, the datasheet of the specific
processor, and the beaglebone SRM to figure them out.
A: There are resources with the muxing options. Please see:
Pinmux Tables
Q: Too many words! Grog angry! Concrete example!
A: Example of an RS232 cape that uses UART2:
/*
* Copyright (C) 2013 Matt Ranostay
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
/dts-v1/;
/plugin/;
/ {
compatible = "ti,beaglebone", "ti,beaglebone-black";
/* identification */
part-number = "BB-BONE-RS232";
version = "00A0";
/* state the resources this cape uses */
exclusive-use =
/* the pin header uses */
"P9.22", /* rs232: uart2_rxd */
"P9.21", /* rs232: uart2_txd */
/* the hardware IP uses */
"uart2";
fragment@0 {
target = <&am33xx_pinmux>;
__overlay__ {
uart_pins: pinmux_uart_pins {
pinctrl-single,pins = <
0x150 0x21 /* spi0_sclk.uart2_rxd |
* MODE1 | PULL_UP */
0x154 0x01 /* spi0_d0.uart2_txd | MODE1 */
>;
};
};
};
fragment@1 {
target = <&uart3>; /* remember the +1 peculiarity */
__overlay__ {
status = "okay";
pinctrl-names = "default";
pinctrl-0 = <&uart_pins>;
};
};
};
This file is located in the kernel sources at
firmware/capes/BB-BONE-RS232-00A0.dts
Compile with:
$ dtc -O dtb -o BB-BONE-RS232-00A0.dtbo -b 0 -@ BB-BONE-RS232-00A0.dts
Copy to /lib/firmware:
# cp BB-BONE-RS232-00A0.dtbo /lib/firmware
Activate:
# echo BB-BONE-RS232 >/sys/devices/bone_capemgr*/slots
Q: I still can’t figure it out! Any pointers out there on how to do stuff?
A: A lot of examples in firmware/capes exist
UART virtual capes: firmware/capes/BB-UART*.dts
I2C virtual capes: firmware/capes/BB-I2C*.dts
SPI virtual capes: firmware/capes/BB-SPI*.dts
# echo BB-UART5 >/sys/devices/bone_capemgr*/slots
Configures UART5, a new /dev/ttyOx device will be created (devices are created sequentially) and will be ready to use.
Similarly
# echo BB-I2C1 >/sys/devices/bone_capemgr*/slots
Creates a new I2C bus master. Note that when you have your own I2C
devices that you need to access, you can either use the user-space I2C
API, or create a device node for your device in the cape. The example
capes have commented out examples on how to do that. All those virtual
capes can be used as a base for your own experimentation.
Q: I still hate Device Tree!!!
A: Sorry, we cannot help you
