Talos™ is a state-of-the-art mainboard designed for the new IBM POWER8 architecture. It is the first and only high performance computer with absolutely no proprietary software or firmware blobs. With its ATX form factor, Talos™ brings a level of performance to the workstation realm usually found in dedicated servers and a level of freedom and user control long extinct in modern hardware. In addition to its onboard, open-toolchain FPGAs, Talos™ easily and tightly interfaces with GPUs, FPGAs, and custom hardware.

Overview

Performance

Talos™ competes with the highest-end mainstream (e.g., x86) computers available - Intel Xeon E5 and even E7 machines. Whether it’s AAA games, intensive CAD and modeling software, or machine learning algorithms for crunching huge datasets, Talos™ is well-equipped to handle the workload:

• Up to 256 GB RAM
• Up to 96 logical cores
• Multiple PCIe, USB, and other interfaces
• Onboard FPGAs

Security

Talos’™ fully open firmware means there are absolutely no inscrutable binary blobs where bugs, backdoors, and vulnerabilities can hide. If you deal with sensitive information or you care about the safety of your intellectual property, Talos™ will dramatically reduce the risk of intrusion and theft with features like:

• Trusted Platform Module (TPM) that verifies all firmware before boot
• No Intel Management Engine or equivalent
• Auditable schematics, firmware, and software
• 100% self-hosting, no need for external tools or compilers

Control

Say goodbye to the days of not being able to configure your system to your needs because of inaccessible firmware, schematics, or toolchains. With Talos™, you own the machine and can modify it to your heart’s content:

• No signing keys preventing firmware modification
• Openly licensed firmware and software (Apache 2.0, GPL 2.0, GPL 3.0)
• Open toolchains for the onboard FPGAs allow for tuning of power sequencing, IRQ routing, and more

Extensibility

You can extend Talos’™ capabilities with hardware accelerators (e.g., FPGAs and GPUs) and with custom peripherals, using a wide array of interfaces:

• GPIO header
• Seven PCIe slots
• Eight USB 3.0 ports

Features & Specifications

• POWER8 Single-Chip Module (SCM) [sold separately]
  • QEMU-supported Hardware Virtual Machine (HVM)
  • Translation Control Entry (TCE), a variant of an IOMMU
  • Vector Multimedia eXtension (VMX)
  • Vector Scalar eXtension (VSX)
  • AES acceleration for VMX / VSX
• Up to 256 GB Memory
  • 8 DDR3 RDIMM slots with ECC support
  • 2 memory controllers
• Peripheral Component Interconnect (PCI)
  • 2 PCIe x16 slots (8 shared lanes)
    • each can become a Coherent Accelerator Processor Interface (CAPI)
  • 4 PCIe x8 slots
  • 1 internal mPCIe 1x slot
  • 1 legacy PCI slot
• Serial ATA (SATA)
  • 8 internal SATA 6 Gbps ports
  • 2 external eSATA 6 Gbps ports
• RS-232
  • 2 external ports with DB-9 connectors
  • 2 internal ports with 10-pin connectors and level shifters
• Universal Serial Bus (USB)
  • 4 external USB 3.0 Type A ports
  • 2 internal USB 3.0 stacked Type A ports
  • 2 internal USB 3.0 ports via a single header
• PS/2 Keyboard and mouse (combined connector)
• General Purpose Input/Output (GPIO)
  • 1 40-pin dual inline header with 0.1" pitch
  • most pins connected directly to one of the onboard FPGAs
• Onboard, open-toolchain FPGAs for controlling low-level operations
• 2 Gigabit Ethernet (GbE) ports
• Video
  • Integrated ASPEED AST2400 for HDMI output
  • Support for discrete GPUs
• Standard ATX form factor
• Heatsink
  • Includes a 92 mm fan
  • Dissipates 190 W continuously in a normal office environment
• Operating System
  • Little Endian Mode
    • Red Hat Enterprise Linux 7.2 or higher
    • SUSE Linux Enterprise Server 12 or higher
    • CentOS 7 or higher
    • Fedora 22 or higher
    • Debian 8.0 ("Jessie") or higher
    • Ubuntu 14.04.3 ("Trusty") or higher
    • Gentoo (planned, build in process)
    • Trisquel (planned for Trisquel 8)
    • others
  • Big Endian Mode
    • Fedora 22 or higher
    • others

The POWER Architecture

Talos™ is designed around IBM’s POWER8 architecture and line of processors. IBM has published a significant amount of detailed POWER8 design and software documentation:

• POWER Instruction Set Architecture (ISA) v2.07
• POWER8 Processor User's Manual for the Single-Chip Module
• IBM's OpenPOWER Connect portal (free account creation required)

OpenPOWER Foundation

Raptor Engineering (a partner of Raptor Computing Systems) is a member of the OpenPOWER Foundation, an open technical membership organization dedicated to the POWER architecture.

A World Beyond x86

The x86 architecture, while ubiquitous in personal computing, suffers from irreparable security and lockdown issues. For example, the Intel Management Engine (ME) is a problem that’s been brewing for about ten years, even though the popular technology press has only recently started reporting on it. The Intel ME is present on all modern Intel processors and is essentially a backdoor with full access to the entire computer - a security disaster waiting to happen. Due to the deeply entrenched interests of current players (e.g., Intel and AMD), and the presumed jungle of legally binding contracts those interests have with their myriad partners, this situation will never improve and will only worsen.

The video below goes into detail about why Talos is a great alternative to x86. (This presentation was first given at the Coreboot conference and developer meeting in San Francisco, June 2016.)

Comparisons & Benchmarks

AnandTech recently did an extensive comparison of IBM’s POWER8 processor with Intel’s Xeon E5. This quote from the conclusion highlights the strength of the POWER8 processor:

The POWER8 microarchitecture is clearly built to run at least two threads. On average, two threads gives a massive 43% performance boost, with further peaks of up to 84%. This is in sharp contrast with Intel’s SMT, which delivers a 18% performance boost with peaks of up to 32%. Taken further, SMT-4 on the POWER8 chip outright doubles its performance compared to single threaded situations in many of the SPEC CPU subtests.

All in all, the maximum throughput of one POWER8 core is about 43% faster than a similar Broadwell-based Xeon E5 v4. Considering that using more cores hardly ever results in perfect scaling, a POWER8 CPU should be able to keep up with a Xeon with 40 to 60% more cores.

Benchmarks and graphs in the remainder of this section are provided as-is for general reference only, and are copyright Raptor Engineering 2016.

High-level Comparison of Machine Architectures

Values are approximate and for illustration purposes only.

The State of General Purpose Computing

Where does Talos™ fit within the larger spectrum of general purpose computers? Talos™ is the only powerful, auditable machine available in a standard ATX workstation / server form factor. Nothing else comes close.

While certain ARM SoCs are relatively open and libre-friendly, their performance leaves much to be desired. Most of these ARM SoCs cannot even compete against x86 hardware from the late 2000s on a raw performance or performance per watt basis. However, they are inexpensive and in some cases suitable for light web browsing or other non-intensive tasks.

On the x86 end, all currently available hardware has been effectively "TiVoized" through vendor- and manufacturer-signed binaries, and can never be audited or modified by the machine owner. Multiple scandals have come to light over the past year involving the manufacturer abusing this position of power to install various forms of malware through a hostile firmware image. In many cases the only recourse was to stop using the computer entirely until the manufacturer released a new firmware version with the malware ostensibly removed. This tradeoff of inexpensive machines for loss of owner control over said machines is unacceptable for many users in the current age of diminishing privacy and increasingly damaging cyberattacks.

Even machines ostensibly built for security are impacted by this situation. For example, the ORWL listed in the table below may have excellent hardware security features, but the software and firmware stack running on its x86 processor is completely unauditable, has full machine control even after operating system launch, and may not be trustworthy. Why would a criminal even bother attacking such a machine physically when a remote wireless or Internet-based exploit would be not only more cost effective but virtually untraceable? Talos™ does not allow such malware or bugs to hide undetected in the most privileged and sensitive areas of your system, namely the firmware and kernel, nor does it ask you to give up modern, high-end general purpose compute performance in the name of security.

Openness & Performance Comparison

• [0] Hardware capable, software still under development
• [1] Lack of open firmware makes owner-controlled trusted boot unavailable
• [2] Multiple closed binary files are required to boot. At least one of them (the ME) is signed by Intel and can never be replaced with a libre alternative.
• [3] We are aware that at least some community members have access to the schematics. Provenance is unknown at this time.
• [4] An attempt is currently being made to activate the IOMMU without blobs. It is still unknown if this is possible on this particular hardware.

High-end Mainboard Comparison

• [0] Although an effort will be made to provide Talos™ schematics to purchasers of the Talos™ mainboard, several pages will be redacted on the public schematic due to mandatory NDAs with a few of our support silicon vendors. The affected areas are primarily related to the PCIe subsystem, do not directly involve the IBM components, and should not hinder third-party development or modification of Talos™ firmware, hardware, or peripherals in most cases. Furthermore, these components do not contain reprogrammable CPU(s), and are also fully isolated behind the IOMMU integrated into the POWER CPU.
• [1] Available only to OpenPOWER Foundation members under a non-disclosure agreement
• [2] Hardware capable, software still under development
• [3] Lack of open firmware makes owner-controlled trusted boot unavailable

Single-Thread Memory Benchmark Comparisons

All benchmarks were run on the actual hardware listed using Raptor Engineering’s modified version of the "bandwidth" utility. You can read a more detailed analysis of these benchmarks in our October 21, 2016 campaign update.

Hynix 16GB x4 ECC RDIMMs operating at DDR3 PC-12800 speeds were used to generate the following memory benchmarks.

POWER8 in Talos™ configuration @ 3.32 GHz

POWER8 in Talos™ configuration @ 3.32 GHz

Intel® Xeon® E3-1270 "Sandy Bridge" @ 3.5 GHz

Intel® Xeon® E3-1270 "Sandy Bridge" @ 3.5 GHz

AMD® Opteron™ 6328 @ 3.2 GHz

AMD® Opteron™ 6328 @ 3.2 GHz

LZ4 Benchmarks (r131)

All benchmarks were run on the actual hardware listed using stock LZ4 sources.

128M binary random file, compression mode

silesia/mozilla, compression mode

Multichase

All benchmarks were run on the actual hardware listed using stock multichase sources. Results are a measure of latency; lower is better.

WARNING: Validity of pingpong results is questionable at this time

OpenSSL

All benchmarks run on actual hardware listed using stock openssl sources.

Demos

Legacy x86 Applications on OpenPOWER™ via QEMU

The following video illustrates a variety of x86_64 native binary applications running on a POWER8 machine using QEMU user mode translation. These applications are using the full 3D hardware capabilities of the host POWER8 machine, and have access to the host’s ALSA audio. Applications that primarily use scalar instructions perform quite well, while applications making heavy use of vector instructions experience lag and stuttering due to missing features in QEMU (see notes below). Multiple 3D games were used to demonstrate the feasibility of a non-native engine binary interfacing with the POWER8 host GPU. At the end of the video, several applications are re-launched using native ppc64el builds for a performance comparison.

Notes:

• Video captured straight from Radeon DVI output.
• ALSA audio functions also, but was not captured for this demonstration.
• This QEMU patch series is required, and was applied to the QEMU master version used in this demonstration.
• Stuttering is largely caused by QEMU not translating target SIMD / vector instructions to host SIMD / vector instructions. QEMU currently translates each vector instruction (e.g. from SSE/SSE2/AVX/NEON/AltiVec) to large chunks of scalar code, resulting in very slow vector execution. This currently affects all target / host pairs available in QEMU, not just the OpenPOWER / x86 combination. The QEMU project is seeking volunteers to implement SIMD / vector translation, so if you would like to help out please contact the developers!
• setreuid() / setegid() calls from target binaries currently crash QEMU. Alien Arena was recompiled for x86 with those two calls commented out for this demonstration.
• Tested using POWER8 with a single AMD Radeon R290X and the open Radeon driver.
• Host kernel must be compiled with 4k page support to match the x86_64 architecture-defined page size.
• DRM passthrough requires that the QEMU host process LP mode match the target LP mode; i.e. i386 targets would require a 32-bit build of QEMU. Because OpenPOWER™ ships with a stock 64-bit kernel and userspace, i386 targets have not been tested.

Unreal® Engine 4 on OpenPOWER™

The following video illustrates the power of Talos™ to accelerate game development. Talos™ allows you to keep your valuable assets and proprietary engine code safe and secure through full owner control, while outperforming similarly priced systems running on the x86 and ARM architectures. Talos™’ massive RAM and GPU bandwidth reduce both compile and asset cook times which frees developers to focus on improving the design and gameplay of their next AAA title.

Notes:

• Radeon 290X used with Mesa 12 in OpenGL 4.3 mode on Debian Strech (LLVM 3.9).
• Flickering directional lights are a general bug in the Unreal® Engine OpenGL backend; this issue is present on all architectures and has been reported to Epic Games.
• Editor load time is similar to load time experienced on the x86 architecture using Linux.
• Video recorded directly from R290X output at 1920x1080 60FPS using 5.1 HDMI audio.
• Several minor dropouts are present in the video due to the extreme bandwidth of the source stream. These dropouts are an artifact of the recording process used and are not visible during gameplay.
• This Unreal® Engine 4 patch
  series is required (agreement with Epic Games required for private access), and was applied to the Unreal® Engine 4 master version used in this demonstration.
• This Unreal Tournament™ patch series is required (agreement with Epic Games required for private access), and was applied to the Unreal Tournament™ master version used in this demonstration.
• This Radeon driver patch is required, and was applied to the radeon driver used in this demonstration.

Building a Complete Talos™ System

You’ll need the following components to build out a complete Talos™ system.

CPU

If you don’t already have one, you can add a POWER8 CPU to your order. We currently offer 8-, 10-, and 12-core options.

RAM

While a full RAM hardware compatibility list (HCL) is not yet available, we can recommend Hynix 16GB x4 ECC RDIMMs operating at DDR3 PC-12800 speeds.

GPU

Although an integrated display driver comes with the Talos™ mainboard, you can also install a discrete GPU of your choosing. We recommend any NVIDIA GK104-based card, which is a reasonably modern GPU that can be initialized by and used with the libre Nouveau driver. Should a discrete GPU be installed, the integrated display driver can either be safely disabled or used as a secondary display output, contingent on software support.

Enclosure

A good choice for an enclosure is the Super Micro Computer SC747TG-R1400B-SQ.

Development Process

Our firmware and software development system for OpenPOWER uses a two-socket reference server from IBM codenamed "Firestone." This allows us to directly verify functionality, check performance, and develop / port new software to Talos™ machines even before Talos™ prototypes have been manufactured. The following images show our test setup.

Angled view of the internals of the Firestone server used for Talos™ prototyping

Top view of the internals of the Firestone server used for Talos™ prototyping

To configure Firestone emulation of Talos™, we pull all cards and disable the secondary CPU package, then we install two memory buffer cards on CP0 and add in PCIe cards containing the new peripherals present on Talos™. While the resultant configuration is not exactly the same as a Talos™ system, and in fact from a pure hardware perspective is quite different, from a performance and software view it is nearly indistinguishable. This configuration is what we use to patch the kernel, modify the OpenPOWER firmware, and gather benchmarks for the Talos™ machines.

Manufacturing Plan

We are in the process of converting the Talos™ designs into the requisite PCB layout. During this process, firmware development is occurring in parallel. Once these two processes are complete, initial bringup will commence, followed by full testing and requisite minor modifications before release for manufacture.

We have many years of experience in all aspects of electrical and software engineering, spanning board level layout through high level application development, without any gaps. As a result, we fully understand each and every part of our designs, the interactions between those parts, and are uniquely situated to handle large, complex projects requiring each component to mesh neatly with the surrounding components, whether those components be hardware, firmware, or software. Our design philosophy is to do it right the first time, and we take explicit steps to ensure that the resultant product will operate in a wide range of adverse conditions. For this particular project we also have the backing of our major silicon vendors, who are willing to work with us to ensure that their devices are used correctly and to their maximum potential.

Risks & Challenges

What could go wrong?

With a project of this complexity, there are numerous potential failure points. Raptor Engineering has significant experience in board level design and system bringup, so we have anticipated and handled the most likely points of failure. That being said, certain unknowns still remain:

• Board space! Cramming a high-performance POWER8 processor and its support circuitry onto a densely populated ATX board means there is zero physical margin in many areas. While we are fairly confident that the board can be routed as depicted, there is still the off chance that some changes to the list of Talos™ peripherals may need to be made during the routing phase.
• The open toolchain for the FPGAs, while apparently stable, has not been tested to the same degree as most commercial offerings. Any number of issues could crop up during bringup of the FPGA section as a result of the toolchain -- our fallback plan would involve developing the FPGA HDL using the proprietary software and assisting the open toolchain project(s) where feasible after Talos™ has shipped.
• DDR3 / PCIe routing / signal integrity issues. Even when design guidelines are rigorously followed, there is always the chance that signal integrity will be compromised somewhere due to nearby signals and components. If this occurs, the fault will need to be isolated and repaired in the prototyping stage.
• System firmware. Current offerings from IBM require the machine owner follow strict guidelines for memory population and related hardware configuration. It is likely that bugs in the IBM firmware will be discovered once these rules are no longer enforced.
• Linux drivers. While overall the switch to little endian has ensured the vast majority of existing Linux drivers function normally on Talos™, there is still the potential for various bugs in these drivers to appear in the final system configuration.

Ordering & Payment

Crowd Supply can accept a wide array of payment options, though credit card is the easiest way to place an order.

Because this is a crowdfunding campaign, any order you place for a Talos™ system using a credit card will not actually be charged until the campaign reaches the funding goal ($3.7M USD). If we don’t reach that goal, no credit cards will be charged and orders paid by means other than credit card will be refunded.

For some product options, you will need to add a POWER8 processor as a separate line item to your order (or order it separately later). These processors are in "pre-order" mode, which means you will be charged for them right away. However, if the campaign does not meet the funding goal, all pre-orders of POWER8 CPUs will be fully refunded.

Why are the CPUs Sold Separately?

Unfortunately, the answer to that question is a little complicated. Basically, it has to do with the complexities of crowdfunding and ensuring that we’re doing all we can to protect backers. Generally speaking, the CPUs are offered separately from the mainboards in order to ensure that the campaign’s funding goal is accurate. While the funding goal is a dollar amount ($3.7M), it is based on a minimum order quantity (MOQ) of Talos mainboards, 900 units. The problem is that basing a campaign on an MOQ is impractical for a number of reasons. For example, backers are unaccustomed to such campaigns, there’s no way to account for support-only pledges, and different components that go into the product can have different MOQs.

Given this reality, the wide price range of CPUs (from $0 if a CPU isn’t wanted, up to $3350 for a 12-core CPU) means it would be basically impossible to accurately or meaningfully set a funding goal. For example, if no one bought a CPU, the goal would be 900 x \$4100 = \$3.7M. On the other hand, if everyone bought a 12-core CPU, then the funding goal would need to be 900 x (\$4100 + \$3350) =$6.7M. Because we can’t possibly predict the demand for each CPU, we therefore need to sell them separately from the mainboards.

You’ll also note that the CPUs are sold in "pre-order" mode as opposed to "crowdfunding" mode, which means your card is charged as soon as the order is placed. That’s because, unlike the mainboards, there is no strict MOQ to meet before they can be manufactured. As with all pre-orders on Crowd Supply, you can cancel and get a full refund at any time. As we note on the CPU page, if the Talos campaign fails to meet its funding goal, all CPU orders will be promptly canceled and refunded in full.

On one hand, this is all a little cumbersome and confusing. On the other hand, it allows you to spread out the cost of purchase over a longer time by giving you the option of purchasing the CPU later. The bottom line is that these processes are in place to best protect backers while maximizing the potential for a successful campaign.