Project update 16 of 36
It’s been a couple months since our last update, in part because the world has been in rapid flux and it’s been a struggle to establish ground truth on a number of production-critical factors. The last time we wrote, the very first doses of the Pfizer vaccine were being administered, while the US Capitol was under assault. Now, about 20% of the US has received at least one dose of a vaccine, and a massive stimulus package is being signed into law.
The supply chain was already squeezed by COVID shutdowns; now it’s getting a double-whammy as large companies are buying and hoarding inventory in anticipation of a “roaring 20’s”-style economic boom, thanks to the large economic stimulus packages being deployed globally. You can see this in reports from Tesla to Honda, where chip shortages are causing production slowdowns in automotive industries. Rumors are that a certain key substrate is forecasted to be in short supply, and in response well-capitalized consumer electronics players are unlocking their cash hoards to pre-purchase allocations through most of this year. Unfortunately, this prevents smaller customers (like Precursor) from gaining access to these precious resources. Furthermore, Bitcoin has increased by 2.5x since the last update; this surge in cryptocurrency valuation is driving a seemingly limitless demand for mining hardware at any price, causing even more pain in the supply chain.
Precursor’s supply chain is feeling the squeeze. Our original production schedule accommodated a 24-week lead time on key parts. Unfortunately, we experienced a series of setbacks acquiring the Xilinx FPGAs, a core and irreplaceable part in the design. Initially, the pricing was tripled from our initial quotations (talk about inflation!); after some negotiation and frantic calls, we finally got something closer to our original price and a lead time of around 40 weeks. However, once we had placed the purchase order and wired money in advance, we heard back that lead time was further pushed out to 50 weeks.
According to our original timeline, we were hoping to start pilot production in July/August 2021. That is no longer realistic. If lead times don’t improve, we’ll be starting production in November, which means the first units may not reach the doorsteps of users until February, instead of the originally anticipated December delivery date.
The semiconductor industry has had to deal with similar problems in the past. After the 2008 economic crisis, it was difficult to purchase linear regulators and op amps because of a critical shortage of high-quality analog chip testers. The industry has also weathered fires in epoxy plants, contamination in photoresists, and geopolitical spats that have caused lead times to push pass the 52-week mark. If the past is any indicator of the future, the long lead times are based on estimates assuming no substitutes are qualified, and no new production capacity comes on line; but typically, within 2-3 months, something changes and we either have qualified alternatives or additional production lines, pulling in lead times. Furthermore, as the winners and losers of the economic recovery become obvious, the losers start to cancel orders, further reducing pressure on the supply chain. Thus, I’m optimistic that we probably won’t see a delay past November, and I will keep pushing to find ways to pull that estimate in.
We’re also evaluating “nuclear options” to pull the production schedule in, including using a higher-power variant of the chip that could be more readily available, but we think in the end the wisest move would be to patiently wait until we can secure authentic chips of the correct speed and power bin for production, even if it takes a few extra months. We’ve also started aggressively purchasing all of the other chips on our BOM, even if they are in stock right now and they would just sit on a shelf until the FPGAs arrive, just so we don’t have an unpleasant surprise later this year and incur further delays to production. Of course, us buying up inventory in the face of decreasing supply contributes to the supply shortage problem, but that’s just how free markets work: absent a system to enforce a fair allocation of scarce resources to the community, the greatest benefits accrue to the shrewdest and fastest buyers.
Our top two choices for the omakase case were stainless steel and naval brass. I’ve spent some time walking around with various Precursor prototype cases in my pocket and shoulderbag, and after a couple of months, we can see how the two material candidates for the omakase case have fared:
On the left is PVD (physical vapor deposition) stainless steel, SUS-304 alloy; on the right is naval brass. Both of them started mirror-shiny and spotless. Before I took this photo, I gave each case a quick wipe-down with isopropanol to remove the fingerprints and light organic grime that had built up. We can see that the PVD steel doesn’t have a scratch. The blemishes on the top left are some more caked-on organic material that would come off with a bit more rubbing. On the right, we can see that the brass has really taken a beating. A set of house keys in the pocket did quite a number on the case backing, and the material is starting to age at an uneven rate; some portions are darker than others. I did try making a version with a brushed finish, and it doesn’t seem to hide the scratches much better, either.
A lot of us were excited about using brass, but I feel the lack of durability is the final strike against the brass – yes, you can polish the brass and recondition it to look quite nice, but I don’t think most users are looking to do that kind of upkeep. Also, the brass is quite conductive; our attempts so far to apply a non-conductive finish to the inside have not been very successful, as they have either delaminated over time or they easily scratch to reveal conductive material underneath. From a safety standpoint, I’d also sleep better at night knowing that the cases were passively safe because they had a robust, non-conductive coating on them, in the case that a battery wire came loose and started contacting the case material.
Thus, I will focus my effort going forward on refining the PVD steel case for the omakase backers. The PVD process itself needs some development to fine-tune the coating. Right now, there are some subtle rippling effects and pores in the coating that need to be fixed by experimenting with different PVD coating parameters. I also want to tighten up some of the dimensions even more; because the steel is so hard, it machines differently from the standard aluminum cases, and I’d like the fit to be just as good on the steel as the aluminum.
That being said, the 7075 aluminum cases are outstanding, and have reached production readiness. Omakase customers can look forward to receiving both versions – an aluminum case, and a PVD steel case; they can swap the internal electronics between the two as they please.
I’ve had repeated requests from the cryptocurrency community to provide a way for users to purchase Precursors using Bitcoin. Thankfully, a volunteer team at Digital Garage has worked out a solution that allows users to purchase devices using Bitcoin, in a way that largely insulates our business operations from the volatility and tax risks of Bitcoin payments. Precursor is now listed in the Digital Garage “Lightning Store”. For complicated reasons, you will find the price listed in Japanese Yen, but at checkout you will only be able to pay with an equivalent value in Bitcoin. Orders placed through this channel will still ship directly from Crowd Supply, and assume the same fulfillment priority and production characteristics as the pre-orderable units available now at Crowd Supply.
We also spent a significant chunk of the past month improving and extending the Xous operating system. As a reminder, we promised to ship the units with a “validation firmware”, which is a single-threaded, non-Xous based demo program, as seen in the backer videos. However, a combination of the projected shipment delays and solid development progress means that we may be able to ship units with an initial, Xous-based firmware.
In a nutshell, Xous is a message-passing microkernel built on a virtual memory model. The kernel is very small—it runs out of a single 4 kiB page of RAM—and the binary shrinks to 23 kiB when debug print statements are turned off. It’s expressly designed to be auditable by individual humans. The kernel handles the allocation of virtual memory to processes, the routing of messages between processes, and the simple round-robin scheduling of time between all the processes and threads. Fast message passing is done by combining Rust-like semantics to borrow, mutably borrow, or move pages of memory between processes. The nature of a borrow, mutable-borrow, or send operation is enforced using the read-only, read-write, or remapping of virtual memory pages between processes. Thus, fairly large data structures can be pushed between processes without physically moving or copying anything; it’s all done by manipulating the page tables, with semantics enforced by the hardware page table mechanism.
Over the past month, we’ve been filling in the services that surround the kernel, including a name server for discovering (and securing) the addresses of other system services, a graphical abstraction layer for approving application requests to draw to the screen, and an input method editor abstraction layer that facilitates predictive text engines. Since the entire system is coded in Rust, our string types natively support UTF-8, and thus almost from day one we were rendering emoji and Chinese characters. Several fundamental hardware abstraction servers have also been rolled out, including the TRNG, keyboard, LCD, embedded controller, and other low-level I/O services.
We’ve finally reached the point where we have a simple “shell” running under Xous. It’s not like your traditional VT-100 style serial console – it’s been re-imaged in the mobile context. Instead of a command line, it uses a chat-bubble style interface, where you send messages to the system as if you were sending a text message, and responses are rendered in a corresponding chat bubble. This was done in part to maximize code re-use toward a key application target of Precursor, which is text-based chat.
Xobs worked hard to make Xous multi-platform from day one, and as a result, you can run our “shellchat” program directly on your desktop using hosted-mode emulation libraries. If you already have the latest "beta" Rust installed on your system, you should be able to clone
https://github.com/betrusted-io/xous-core/ and type
cargo xtask run. This will build all the code from source, and pop up an emulated version of the Precursor screen. Note that we need the beta toolchain, as we are using
min_const_generics, which becomes stable in release 1.51 (scheduled for March 25th). Until then, you’ll need to add the
+beta flag to run Xous if you’re using stable Rust.
Keep in mind the emulation is fairly slow, so typing input to the system requires a bit of patience, but it’s good enough to do some simple application framework development and debugging. Fortunately, performance on the actual hardware is much faster.
We’ve also been working hard to improve the documentation resources for Xous, and have collected most of our documentation in a centralized location: https://github.com/betrusted-io/betrusted-wiki/wiki. We invite you to have a look around, to open issues on our GitHub repo, or to ask questions in our IRC channel at
Our next priority is to finalize our hardware prototypes so that we can secure the FCC and CE certificates necessary to legally export Precursor. This will be a challenging, in part because the test requires extensive interaction with the testing facility and the rules around international travel are still in flux. We’re also building up our production assembly and test infrastructure. Developing a fast, comprehensive, easy-to-use test for the factory is a formidable challenge. So far we’ve fabricated a few custom circuit boards to support the activity, developed custom mechanical jigs, and written a fair chunk of code to back it up. However, there’s a glimmer of hope that we may be able to travel on-site to our factory in South Korea before the end of the year to install the infrastructure in person, instead of trying to do a line bring-up by Zoom. Finally, we’re going to keep pushing ahead with layering in more and more features into Xous – the shellchat demo is just the head of the trail for Xous.
Stay safe, and happy hacking!
-bunnie & Xobs