Vision FPGA SoM

by tinyVision.ai

An FPGA-based SoM with integrated vision, audio, and motion-sensing capability

$1,918 raised

of $8,000 goal

24% Funded

Pledge Now

$70

FPGA Vision SoM

One FPGA Vision SoM to play with! The price will increase once the campaign is over, so back us today!


$80

Vision FPGA SoM Breakout Kit

One Vision FPGA SoM and one Breakout Board.


$120

Vision FPGA SoM Dev Kit

One Vision FPGA SoM and Developer Board.


$125

Two FPGA SoM

Two FPGA Vision SoM's.


$620

Ten FPGA SoM

Pack of 10 FPGA Vision SoM's, at a 5% discount.


$14

SoM Breakout Board

The SoM breakout board brings all SoM pins to 2.54 mm pitch pins for breadboarding, can't do without it! SoMe assembly required to solder the 2.54 mm headers on. Note: SoM is not included.


$60

SoM Developer Board

One developer board that breaks out SoM IO with in-line LEDs, allows for USB power (with power monitoring) and programming, includes Pmod and Qwiic connectors, and has room for prototyping. Note: SoM is NOT included.

Details

Recent Updates


As Featured In

CNXSoft - Embedded Systems News

"Vision FPGA SoM Integrates Audio, Vision and Motion-Sensing with Lattice iCE40 FPGA"

Low-power Computer Vision

Embedded Computer Vision is a highly multi-disciplinary field that requires expertise in optics, image sensors, hardware, firmware and so on. As a result, the bar to playing with technology in this field is quite high.

The Vision FPGA SoM is a Lattice FPGA based System on Module that integrates an ultra-low-power vision sensor, a 3-axis accelerometer/gyroscope, and an I²S MEMS microphone in a small form-factor (3 cm x 2 cm).

This device is built for developers who want to not only play with this technology, but also have a path to integrating it into real world products. We designed the device with the following thoughts in mind:

  • Low Power Image Sensor + FPGA: Overall power consumption is minimized and in the 10-20 mW range to enable battery powered applications.
  • Well-designed Host Interface: Easy to integrate into a larger system using an interrupt driven SPI host interface, programmable IO voltages to support a glueless HW interface.
  • Modular Design: Complexity will be localized to the SoM so that the developer can quickly integrate this device into their system using a breadboard as well as a simple API. All required components for vision/audio/motion (eg. image sensor and illumination) will be integrated so the developer doesn't have to cobble together parts. The solution translates into a final product with minimal changes.
  • Flexible and Easy to Use: Easy to integrate mechanically and electrically. The SoM will appear like an SPI device with a SW library to support it. The user has the ability to configure the FPGA CRAM instead of having to program the flash followed by a reset to configure the FPGA. The developer has the flexibility to choosing an image sensor while providing a reasonable, low-cost default option.
  • Strong Set of Features: Vision FPGA SoM is not just an FPGA board. Require a reasonably sized SRAM to allow for temporary data storage at low power (not DRAM!) which is especially important for vision applications where multiple image frames may need to be captured/processed. Audio and IMU are commonly used together with vision and will make sense to integrate on this platform.
  • Documented and Open FPGA Code: The SoM supports an open source toolchain. Well-documented code and plenty of simple examples for each subsystem exists, as well as a large design that ties together various parts of the SoM.
  • Production Capable: The SoM form-factor and connectorized interface enable a smooth transition from prototype to production.

The modular concept is summarized in a talk given at a tinyML meetup in 2019.

Flexible and Tightly Integrated

By including most commonly used sensors in a configurable platform utilizing an FPGA with an open source toolchain, this device enables developers to experiment and build quickly. Sample FPGA, as well as host code will be provided as a jumping off point for backers.

Vision FPGA SoM top.

Features & Specifications

  • The main processing element is the Lattice iCE40UP5k FPGA
    • 5K LUT's
    • 1 Mb RAM
    • 8 MAC units
  • Main Onboard Sensor: Color, rolling shutter imager (Himax HM01B0)
  • Other Onboard Sensors:
    • Knowles MEMS I²S microphone, expandable to a stereo configuration with an off-board I²S microphone
    • InvenSense IMU 60289 6-axis Gyro/accelerometer
  • Memory:
    • 4 Mb qSPI Flash for FPGA bitstream/code storage
    • 64 Mb qSPI SRAM for temporary data
  • LED's:
    • Tri-colour LED for a user interface driven by the FPGA
    • IR LED for low-light illumination with frame exposure synchronization
  • GPIO:
    • Four GPIO, each with programmable IO voltage
    • Four-wire SPI host interface with programmable IO voltage
  • Flexible Power Options:
    • Single 3.3 V operation, can supply 1.8 V and 1.2 V at 100 mA (max) to external devices using onboard LDO's
    • External 3.3 V, 1.8 V, 1.2 V for lower power operation
  • Neural Network Support: Supports the Lattice SensAI toolchain using Tensorflow/Caffe/Keras for model development, quantization, and mapping to the SensAI Neural Network engines. Inference at >30 FPS at ~25 mW (average). Sample FPGA code provided for:
    • Vision-based people detection
    • Audio keyword detection
  • Small size: 2 cm x 3 cm

Accessories for Development and Prototyping

SoM Developer Board

The Developer Board provides the following functionality that enables application development using the module:

  • Brings out all IO to headers
  • Micro USB based power and FPGA programming
  • LED's on all GPIO for signal monitoring
  • Module power monitoring to allow for application development
  • Standard headers (dual Pmod and Qwiic) for attaching expansion boards such as Wi-Fi/BLE and other sensors for quick prototyping
  • Small prototyping area

Vision FPGA SoM on the SoM developer board.

SoM Breakout Board

The Breakout Board brings out all the pins on the SoM to 2.54 mm pitch breadboard friendly format to allow you to integrate the SoM into your project. Note that the FPGA must be programmed using an external SPI flash programmer (programming supported using a FT232 USB adapter, Raspberry Pi, or microcontroller among other options).

SoM breakout board.

Comparisons

ItemFPGA Vision SoMiCEBreakerTinyFPGA BXTomu FPGA
OSHW? OSHWOSHWOSHWOSHWClosed
Price $70$69$38$45$49.49
Schematics Published? YesYesYesNot yetYes
Design Files Published? Not YetYesYesNot yetNo
Volume Production Friendly? YesNoNoNoNo
FPGA
FPGA Model iCE40UP5KiCE40UP5KiCE40LP8KiCE40UP5KiCE40HX1K
Logic Capacity (LUTs) 52805280768052801280
Internal RAM (bits) 120k + 1024k120k + 1024k128k120k + 1024k64k
Multipliers 88080
Peripherals
USB Interface FTDI 2232HQ (dev kit)FTDI 2232HQOn FPGA bootloaderOn FPGA bootloaderFTDI 2232HL
User IOs 827 + 741 + 24 + 218
Pmod Connectors 1 (dev board)3001
User Buttons 01 tactile + 3 tactile on breakoff Pmod1 reset2 capacitive0
User LED 1 tricolor, high-power IR LED2 on-board4 on dev board2 + 5 on breakoff Pmod5
Onboard Clock 12 MHz, shared with FTDI12 MHz, shared with FTDI16 MHz12 MHz12 MHz, shared with FTDI
Flash 8 Mb QSPI128 Mbit QSPI DDR8 Mbit SPI16 Mbit SPI32 Mbit SPI
SRAM 64 Mbit QSPI PSRAMNoNoNoNo
IMU InvenSense 6 DOF (accellerometer + gyroscope)NoNoNoNo
Mic MEMS I²S microphoneNoNoNoNo
Power Measurement On dev kitNoNoNoNo
Open Source Toolchain YesYesYesYesYes
APIO YesYesYesYesYes
Icestudio YesYesYesNot yetYes
Migen NoYesYesNoYes

Support & Documentation

All data about the SoM is captured in the Vision FPGA SoM GitHub repo and will be updated over time with sample code, Colab notebooks, etc.

Manufacturing Plan

Once the campaign concludes successfully, we will initiate completion of the verification run for the SoM, developer kit and breakout boards, followed by thorough testing to verify functionality. We have a couple of assembly vendors already lined up and will be negotiating the final assembly pricing in parallel to the design effort. The final delivery dates for the devices will be dialed in based on the lead-times for parts and agreed upon build schedules with the assembler and communicated to backers. PWB’s will be ordered from ALLPCB as in previous runs followed by assembly and test and fulfilled by Crowd Supply.

Fulfillment & Logistics

The devices will be manufactured/tested by tinyVision.ai Inc. and bulk shipped to Crowd Supply from where they will be shipped to the backers. For more information, you can refer to this useful guide to ordering, paying, and shipping.

Risks & Challenges

Some sources of risk are:

  • Design: Prototypes have been built and tested for functionality.

    • SoM: The SoM functionality has been verified for the most part. The prototype was designed for both a global shutter imager and a rolling shutter imager for flexibility. Feedback from a survey indicates that most potential backers would prefer a color imager and don't care as much for rolling vs. global shutter. This change needs to be made and a final pre-production run will be required to verify the design.
    • Developer Board: The current prototype is overly complex in that it adds a large number of level translators when not needed. This has been significantly simplified and a final design has been completed. This needs to be built and tested.
    • Breakout Board: The prototype board is functional electrically. However, once the 2.54 mm pitch pins were soldered on, we realized that these would short very easily to the SoM. Design changes are being made to avoid this issue.
  • Manufacturing:

    • The builtin image sensors are chosen for their low power profile. However, these devices are not commonly available. We have built up good relationships with the suppliers and also obtained quotes from the vendors in anticipation along with lead times.
    • As with any hardware product, there are challenges of maintaining quality and testing. We are not new to manufacturing products for customers: we have been building and selling the UPduino and Himax adapter boards for a couple of years.

That said, if there are any delays or complications with manufacturing or delivery, we will communicate this clearly and promptly with backers.

You can learn more about how Crowd Supply protects it’s backers in their Guide.

Funding ends on Oct 02, 2020 at 04:59 PM PDT (11:59 PM UTC)


Credits

tinyVision.ai

tinyVision.ai works closely with clients to incorporate low power CV into their devices. We are enabling CV with hardware in the form of tightly integrated CV modules with a clean API.


Venkat Rangan


Emcraft Systems

Manufacturer

Pixart Imaging

Imaging

Himax

Component provider

Subscribe to the Crowd Supply newsletter, highlighting the latest creators and projects: