Structure SDK (Cross-Platform) Release Notes

Structure SDK (Cross-Platform) 0.9

Released May 4, 2021

We are excited to announce our team has completed development and validation of Structure SDK (Cross-Platform) 0.9. Our team focused this release on extended firmware version support, better error handling, and the addition of a Dark Object Preset (beta).

Improved:

  • Revised error handling when firmware and drivers are mismatched - now you can use Fwupdater to detect if your sensor firmware is obsolete.
  • Structure SDK (Cross-Platform) 0.9 is now compatible with all sensors running firmware 1.0 or newer.

New

  • Added a new feature to receive the current running state of dynamic calibration
  • CoreFirmwareUpdater now has the ability to resolve a sensor state of “Corrupt Production Sync”, restoring your sensor to working faster and without the need to contact support.
  • CoreFirmwareUpdater can now also downgrade a sensor’s firmware.
  • Dark Object Preset (Beta) is available for all sensors running firmware 1.1 and higher. The Dark Object Mode increases the depth map coverage at the expense of depth accuracy. This is expected to be helpful in machine vision applications that require sensing of dark and shiny objects. Test the improved Dark Object Preset in CorePlayground today and see if it fits your needs!
  • Support for Firmware 1.2

Firmware 1.2 Release Notes:

  • Resolved an issue where Structure Sensor Mark II was unable to communicate with certain iPad models due to a change by Apple to USB-C authentication
  • Added a new Dark Object Preset that enables improved depth in scenes containing dark, shiny objects (e.g. computer monitors, luggage). Please, note that an iOS version of this preset is coming soon.

Structure SDK (Cross-Platform) 0.8.1

Released Oct 6, 2020

This release requires Structure Core firmware version 1.0.0.

Firmware and SDK

  • Fixed an issue that one-shot dynamic calibration randomly caused the sensor disconnection.
  • The timeout of one-shot dynamic calibration extends to 10~15sec to get the better result.
  • Fixed an issue that sensorInfo content is empty when connecting a sensor.
  • Now the sensorInfo structure in CaptureSession can return sensor information even if the firmware version is mismatched.
  • Reinforced the stream reliability across all platforms.
  • Disabled the visible frame chunking for visible frame integrity.
  • Revised some under-the-hood debug message.

Samples:

  • Disabled the IMU by default in CorePlayground. You can still enable it in Configuration GUI.

How to upgrade/downgrade your sensor:

NOTE: Please DO NOT update the host USB driver on your PC until after upgrading your sensor. Due to certain changes in firmware, sensors running older firmware require the older USB drivers to connect.

If you are using SDK 0.7.3 or earlier, please follow the following steps to upgrade your sensor to latest firmware.

  1. Plug in the sensor.
  2. Open DriverAndFirmware/ <platform> /CoreFirmwareUpdater </platform> .
  3. Click the upgrade button, and wait for “Update succeeded” message.
  4. Re-plug the sensor.
  5. If you are running Linux host, open terminal and then run the command sudo DriverAndFirmware/Linux/1_0_0_Driver/Install-CoreDriver-Udev-Linux.sh
  6. If you are running Windows host, install DriverAndFirmware/Windows/1_0_0_Driver/Windows/OCDriverInstaller.exe. Select y (yes) to proceed with the driver installation and press exit if the process is finished. Otherwise, you don’t have to do anything.
  7. Now your host is ready to communicate with the Structure Core sensor with the latest firmware and start to work on latest SDK. Please note that once your sensor is upgraded, it won’t be compatible with 0.7.3 SDK or before. If you have the downgrade request, please contact Structure for support.

Note: If your sensor failed to upgrade on Linux, try to run the command sudo DriverAndFirmware/Linux/0_9_0_Driver/Install-CoreDriver-Udev-Linux.sh. Then, replug the sensor and launch DriverAndFirmware/ <platform> /CoreFirmwareUpdater </platform> again.

If your sensor failed to upgrade the sensor on Windows, try to run the command sudo DriverAndFirmware/Windows/0_9_0_Driver/Install-CoreDriver-Win64.exe. Then, replug the sensor, and launch DriverAndFirmware/ <platform> /CoreFirmwareUpdater </platform> again.

Stream with Core Playground, DepthTester, and MultiRecorder (Cross-Platform)

This package contains the release of Core Playground, DepthTester, and MultiRecorder open-source samples.

  • CorePlayground is a tool that enables testing and tweaking of streaming functionality on Structure Core. You can enable streams for every sensor on the device and adjust their settings.
  • DepthTester includes host-processing depth functionality for experimental testing, for example, evaluation of the depth performance on dark surface and reflective objects.
  • MultiRecorder shows multi-sensor streaming functionality.

Known issues

  • Some deprecated API interfaces remains in header files.
  • CorePlayground Android version sometimes doesn’t fight for “Structure Core access permission”. It will require to re-run the app and re-plug the sensor.

Structure SDK (Cross-Platform) 0.8

Released May 19, 2020

This release requires Structure Core firmware version 1.0.0.

If your sensor has firmware version 0.9.2~0.9.7, please follow the instructions posted below to upgrade your sensor. If your firmware version is older than 0.9.2 (i.e. 0.8.6), please contact Structure for support in upgrading your sensor.

Firmware and SDK

  • Streaming from multiple sensors is now supported. For more in-depth information, as well as example code, look at the new open-source sample Multi-Recorder
  • Structure Core sensors with QVGA depth enabled now support native QVGA depth streaming. This option is enabled by default when QVGA is provided as the resolution to CaptureSession
  • Depth frames can now be streamed at 60 frames-per-second
  • Fixed a bug with temperature calculation for Structure Core sensors. Temperature, reported in C, should now be more reliable
  • Fixed the “timed allocation error” bug which affected all previous releases, including FW 0.9.7

Samples

  • Fixed finger scrolling on the main UI for Android applications, no longer requiring the scrollbar
  • Color-demosaic algorithm can now be selected as an option in the CorePlayground GUI

How to upgrade your sensor

NOTE: Please DO NOT update the host USB driver on your PC until after upgrading your sensor. Due to certain changes in firmware, sensors running older firmware require the older USB drivers to connect.

If you are using SDK 0.7.3 or earlier, please follow the following steps to upgrade your sensor to latest firmware.

  1. Plug in the sensor. Open DriverAndFirmware/ <platform> /CoreFirmwareUpdater </platform> , click the upgrade button, and wait for “Update succeeded” message.
  2. Re-plug the sensor.

If you are running  Linux host, open terminal and then run the command sudo DriverAndFirmware/Linux/1_0_0_Driver/Install-CoreDriver-Udev-Linux.sh.

If you are running  Windows host, install DriverAndFirmware/Windows/1_0_0_Driver/Windows/OCDriverInstaller.exe. Select y (yes) to proceed with the driver installation and press exit if the process is finished.

Otherwise, you don’t have to do anything. Now your host is ready to communicate with the Structure Core sensor with the latest firmware and start to work on latest SDK.

Please note that once your sensor is upgraded, it won’t be compatible with 0.7.3 SDK or before.

NOTE: If your sensor failed to upgrade on Linux, try to run the command sudo DriverAndFirmware/Linux/0_9_0_Driver/Install-CoreDriver-Udev-Linux.sh, replug the sensor, and launch DriverAndFirmware/ <platform> /CoreFirmwareUpdater </platform>  again.

If your sensor failed to upgrade the sensor on Windows, try to launch DriverAndFirmware/Windows/0_9_0_Driver/Install-CoreDriver-Win64.exe, replug the sensor, and launch  DriverAndFirmware/ <platform> /CoreFirmwareUpdater </platform> again.

Stream with CorePlayground, DepthTester, and MultiRecorder (Cross-Platform)

This package contains the release of Core Playground, DepthTester, and the new MultiRecorder open-source samples.

  • CorePlayground is a tool that enables testing and tweaking of streaming functionality on Structure Core. You can enable streams for every sensor on the device and adjust their settings
  • DepthTester includes host-processing depth functionality for experimental testing, for example, evaluation of the depth performance on dark surface and reflective objects
  • MultiRecorder shows multi-sensor streaming functionality

Known issues

  • Sensors will disconnect occasionally on Windows when streaming from multiple sensors
  • Sensors will disconnect occasionally when running one-shot calibration
  • Old, deprecated functionality has been removed from certain headers, namely CaptureSessionTypes.h

Structure SDK (Cross-Platform) 0.7.3

Stream with Core Playground (for all platforms)
This package contains the official cross-platform release of CorePlayground, a tool that enables testing and tweaking of streaming functionality on Structure Core. You can enable streams for every sensor on the device and adjust their settings.

This version of CorePlayground replaces the previous macOS-only preview release. Source code is included. Also, pre-built cross-platform Core Playground binaries have been added under Samples/PreBuilt now.

ROS1 Driver and ROS2 Driver Beta (for Linux only)

This package also contains ROS1 Driver and ROS2 Driver beta, which allows users working on Robotics OS to integrate the Core SDK seamlessly. ROS2 Driver Beta is still under development and we are looking forward to all of your feedback here on the community forums or via email developers@structure.io

Develop with Structure SDK

This package contains Structure SDK for Windows, Linux, macOS, and Android as well as ROS Driver for Linux. You can find the full release notes for Structure SDK 0.7.3 (Cross-Platform) ROS Driver below.

SDK

  • Added convertDepthToUShortInMillimeters function for DepthFrame to return the depth values as unsigned short integers in millimeters.
  • Added temperature variable in CaptureSessionSensorInfo to hold the die temperature of the sensor.
  • Added projectorPowerEnabled variable in CaptureSessionSettings to enable/disable the laser projector when streaming starts.
  • Added getCameraType function for CaptureSession to return the camera type (color or mono one) of the Structure Core sensor.
  • Fixed a bug in DepthFrame::operator() . Elements were not being accessed correctly.
  • Fixed a bug with the CameraFrames::getImuFromVisibleExtrinsics function. The returned extrinsics value was not correct.
  • Fixed a bug that prevented users from running CaptureSession in a Windows service application.
  • Improved depth correction performance.
  • Removed 800Hz as an option in StructureCoreIMUUpdateRate . The new values are 100Hz, 200Hz, 400Hz, and 1000Hz to better reflect what the Structure Core sensor supports.

Structure ROS Driver (x86_64 and arm64 Linux only)

ROS 1 changes

  • Added Depth to Point Cloud conversion node, including the option to publish without NAN values.
  • Added support for multiple Structure Core connections using the Structure Core serial number.
  • Added option for setting the IR Publish framerate different from Core streaming capture rate.
  • Added IMU Node - Frame ID.
  • Added example sample for using PCL.
  • Added example sample for using OpenCV.
  • Added example sample for using ROS depth_image_proc package in order to construct RGB-D point cloud.
  • Added parameter rviz_frame in order to toggle pointcloud orientation in rvid application.
  • Changed IMU Coordinate System to Right-Hand.
  • Changed Point Cloud to Optical Frame Coordinates.
  • Increased IR BothCameras publisher speed.
  • Increased Depth publisher speed.
  • Fixed rtabmap initialization error in rta
  • Fixed ROS Frame timestamp to match Core Exposure timestamp.
    bmap integration.
  • Fixed SC namespace so that it can operate in nodelet.
  • Fixed pointcloud node names and namespaces so they can work with multiple cameras at the same time.
  • Fixed sensor serial number type.
  • Fixed an issue where setting the initial auto-exp sometimes failed on ARM boards.

Known issues

  • CaptureSessionUSBVersion will always return USB1 if the USB port in use is not USB3.
  • When streaming on multiple sensors over a USB2 port with FrameSync enabled, some frames may not be delivered. (Such as VGA Depth + Infrared, both 30FPS)
  • The CPU consumption is still high while running ROS Driver or other sample applications on Linux.

Structure SDK (Cross-Platform) 0.7.2

Released July 10, 2019; minor update on Dec 19, 2019

We’re announcing yet another Structure SDK (Cross-Platform) 0.7.2. This is also another follow-up maintenance update to the SDK.

This release required firmware version 0.9.7. If your sensor has firmware version 0.9.3 to 0.9.6, you can use the included CoreFirmwareUpdater tool to upgrade. If your firmware version is older than 0.9.2 (i.e. 0.8.6), please contact Structure for support in upgrading your sensor.

Release type: Windows, Linux, macOS, Android
Firmware: Version 0.9.6 required
Architectures: x86_64 (Windows/Linux/macOS), arm64 (Linux/Android)

API changes

  • Fixed a bug in streaming where starting->stopping->starting would never work
  • Fixed a bug in XRSession::readLastVisibleFrame where our rendering+savePNG functions would show cbcr artifacts

ROS Driver beta (for Linux)

This package also contains ROS Driver beta, which can allow users working on Robotics OS to integrate Core SDK seamlessly. We are still making ROS Driver get better and welcome all feedbacks!

  • Fixed and revised documentation for both ROS and ROS2 Drivers that can be found in Documentation/html/index.html
  • ROS Driver now reports Frame timestamp from Structure Core instead of ROS host time

Known issues

  • CaptureSessionUSBVersion will always return USB1 if the USB port in use is not USB3.
  • When streaming on multiple sensors over a USB2 port with FrameSync enabled, some frames may not be delivered. (Such as VGA Depth + Infrared, both 30FPS)
  • The CPU consumption is still high while running ROS Driver or other sample applications on Linux.
  • ROS Driver cannot run well with structure core connecting with some USB extention cables

Update 0.7.2 (December 2019):

We have made a small update to the SDK to include the ROS Driver beta, which allows users working on Robotics OS to integrate the Core Cross-Platform SDK seamlessly. We are still working to improve the ROS Driver, and thus welcome all your feedback!

Structure SDK (Cross-Platform) 0.7.1

Released Apr 22, 2019

We’re announcing Structure SDK (Cross-Platform) 0.7.1 - a follow-up maintenance update to Structure SDK (Cross-Platform) 0.7, which is considered a major release.

This release requires Structure Core firmware version 0.9.6. If your sensor has firmware version 0.9.3 or 0.9.4, you can use the included CoreFirmwareUpdater tool to upgrade. If your firmware version is older than 0.9.2 (i.e. 0.8.6), please contact Structure for support in upgrading your sensor.

Release type: Windows, Linux, macOS, Android
Firmware: Version 0.9.6 required
Architectures: x86_64 (Windows/Linux/macOS), arm64 (Linux/Android)

API changes

  • StructureCoreDepthRangeMode presets have been reconfigured internally to work better across Structure Core devices.
  • Added convertDepthToRgba function for DepthFrame to help with rendering depth frames using a color map.
  • Added saveImageAsPointCloudMesh function for DepthFrame. Outputs a gray, 3D point cloud as a PLY mesh.
  • DepthFrame::saveImageToPngFile now outputs a 16bit PNG image with each pixel holding the depth value in millimeters.
  • Additional documentation for saveImageToPngFile functions for all CameraFrames types. Details output pixel format and image type.
  • The default StructureCoreDepthResolution enum is now VGA instead of QVGA.
  • Added BodyScanning enum preset to StructureCoreDepthRangeMode.
  • Added minMaxDepthInMmOfDepthRangeMode to CaptureSessionSettings. Returns the estimated min and max depth values possible for a specified StructureCoreDepthRangeMode preset.
  • Added Streaming enum to CaptureSessionEventId. Useful for registering when the internal driver has officially started streaming frames.
  • Added createDirectories function to Utilities.h. Used to create directories when they do not exist for each platform.

Windows

  • The SDK can now detect and report when the Windows USB Driver for Structure Core is not installed.
  • Added USBDriverNotInstalled enum to CaptureSessionEventId to report out when the Windows USB Driver for Structure Core is detected to not be installed.

CorePlayground

  • applyExpensiveCorrection for DepthFrame is now on by default.
  • CaptureSessionUSBVersion is now displayed in the Streaming GUI.
  • Full OCC recording is now supported. Click on the “Record OCC” button in the Streaming GUI to start, and again to stop. The output OCC file will be in your Documents folder in Documents/occ.
  • A button to export a snapshot on all streaming sensor, Save Images & PLY, has been created. The output images will be in your Documents folder in Documents/occ.
  • The color map table when rendering depth frames now uses the estimated min and max depth of the specified StructureCoreDepthRangeMode.
  • When a StructureCoreDepthRangeMode is specified, the exposure and gain UI sliders for infrared frames now correctly adjust to the presets internal values.
  • When One-shot calibration is enabled, it will no longer be enabled when returning to the Configuration GUI from the Streaming GUI after streaming is stopped.

Known issues

  • CaptureSessionUSBVersion will always return USB1 if the USB port in use is not USB3.
  • When streaming on multiple sensors over a USB2 port with FrameSync enabled, some frames may not be delivered. (Such as VGA Depth + Infrared, both 30FPS)

Structure SDK (Cross-Platform) 0.7

Released Mar 18, 2019

We’re super excited to announce the release of Structure SDK (Cross-Platform) 0.7!

This is our first major update to the Structure SDK (Cross-Platform) since releasing Structure Core. It now features a  unified API across all supported platforms, and adds full macOS support.

With this release, we have focused on a number of different improvements to both the firmware found within Structure Core and the software it interacts with.

These improvements include dynamic calibration, improved color frame conversion with options, and a cross-platform core playground sample application.

Full Release Notes

This release requires Structure Core firmware version 0.9.4. If your sensor has firmware version 0.9.2 or 0.9.3, you can use the included CoreFirmwareUpdater tool to upgrade.

  • Release type: Windows, Linux, macOS (new), Android
  • Firmware: version 0.9.4 required
  • Architectures: x86_64 (Windows/Linux/macOS), arm64 (Linux/Android)

If your firmware version is older than 0.9.2, please contact support@structure.io for support in upgrading your sensor.

Dynamic Calibration (Beta)

Dynamic Calibration can recover depth in a situation where the sensor is deformed due to mounting, impact or other causes.

edited

Improved Color

With the new demosaic option, all Structure Core (Color) will see improved color. These improvements are made by making improvements to the color pipeline.

Cross-Platform CorePlayground

This package contains the official cross-platform release of CorePlayground, a tool that enables testing and tweaking of streaming functionality on Structure Core. You can enable streams for every sensor on the device and adjust their settings.

This version of CorePlayground replaces the previous macOS-only preview release.

API changes

  • Added getImuFromDepthExtrinics and getImuFromVisibleExtrinsics functions to CaptureSession.
  • Added support for Structure Core dynamic infrared calibration. See StructureCoreDynamicCalibrationMode. This is a beta feature and is disabled by default.
  • Added options for Structure Core color frame conversion. See StructureCoreDemosaicMethod.
  • Added default values for StructureCoreVisibleResolution and StructureCoreInfraredResolution.
  • Structure Core visible frame gamma correction is now enabled by default.
  • Added saveImageToPngFile function for DepthFrame, ColorFrame, and InfraredFrame.
  • ConnectedSensorInfo and enumerateConnectedSensors() are no longer defined in terms of C++ STL types. This resolves a crash on Windows when using enumerateConnectedSensors() in combination with the C++ multithreaded debug runtime ( /MDd option to cl.exe ).
  • References to functionality specific to iOS and Structure Sensor have been removed.
  • The colorFrame member of CaptureSessionSample has been renamed to externalColorFrame .
  • The MathTypes.hpp header has been merged into MathTypes.h .
  • CaptureSession::sensorSerialNumber() is deprecated in favor of CaptureSession::sensorInfo() which returns additional information.
  • GL.h header has been removed.
  • Removed iOS-specific IMU data types from IMUEvents.h .
  • Removed iOS-specific functionality from OCCFileWriter.h .
  • Removed unused functionality from Utilities.h .
  • Added Android-specific function registerSensorByUSBFileDescriptor() which replaces USB device file descriptor and path options in CaptureSessionSettings.
  • Additional documentation for CaptureSessionUSBVersion.

All platforms

  • All platforms now share the same sample codebase and are built with CMake.
  • CorePlayground is now cross-platform and includes source code. It replaces the platform-specific Recorder sample apps from previous SDK versions.
  • The Linux SimpleStreamer example is now available for all platforms.

Windows

  • The size of the Structure library has been reduced by over 50%.

macOS

  • Full support for macOS has been added.

Android

  • The size of the Structure library has been reduced by over 50%.
  • The auxiliary libraries libAndroidUtils.a , libceres.so , and libusb1.0.so have been removed from the SDK.

Known issues

  • Structure Core devices may reboot continuously if connected to a standby charging USB port (often marked with a lightning bolt or battery symbol). A powered USB hub or a different port should be used instead until this issue has been fixed.
  • Camera and IMU streams may fail to initialize when repeatedly starting and stopping the streams.

Structure SDK (Cross-Platform) 0.6.1

Released Feb 1, 2019

We’re announcing Structure SDK (Cross-Platform) 0.6.1, a follow-up update to Structure SDK (Cross-Platform) 0.6.

This new SDK version requires Structure Core Firmware 0.9.3.

This release requires Structure Core firmware version 0.9.3. If your sensor has firmware version 0.9.2 please use the included firmware updater to update your sensor’s firmware. If you have firmware 0.8.X please contact support@structure.io

Structure SDK (Cross-Platform) 0.6

Released Dec 13, 2018

The Structure team has been working hard to get Structure SDK (Cross-Platform) ready for use with Structure Core, and we’re excited to say it’s available for download!

Structure SDK (Cross-Platform) 0.6 includes the following:

  • Structure SDK for Windows
  • Structure SDK for Android
  • Structure SDK for Linux (x64, ARM)
  • Core Playground (preview release, for macOS only)

Stream with Core Playground (for macOS)

This package contains a preview release of Core Playground, a tool that enables testing and tweaking of streaming functionality on Structure Core. You can enable streams for every sensor on the device, adjust their settings, and save frame data. If you have access to a Mac, we highly recommend starting with Core Playground to familiarize yourself with Structure Core.

For a more in-depth overview of Core Playground, please watch this Structure webinar.

Develop with Structure SDK

These packages contain Structure SDK for Windows, Linux, and Android. Within these SDK packages, you’ll receive reference material (README.html) along with the development tools for the different supported platforms: Android, Linux, and Windows.

Setting up a program to stream data directly from Structure Core is as easy as writing a couple dozen lines of code, as seen below:

struct MyCaptureSessionDelegate : public ST::CaptureSessionDelegate {
  void captureSessionDidOutputSample (CaptureSession* session, const CaptureSessionSample& sample) override {     
    switch (sample.type) {
       case ST::CaptureSessionSample::Type::SynchronizedFrames:
         // we have sync'd frames!
         // ST::Matrix4 cameraPose = ST::Matrix4::nan();
         // bridgeEngine.provideRgbdFrame(sample.depthFrame, sample.visibleFrame, cameraPose);
         break;
       case ST::CaptureSessionSample::Type::AccelerometerEvent:
         // we have accel data!
         // bridgeEngine.provideAccelData(sample.accelerometerEvent);
         break;
       case ST::CaptureSessionSample::Type::GyroscopeEvent:
         // we have gyro data!
         // bridgeEngine.provideGyroData(sample.gyroscopeEvent);
         break;
     };
   }
 };

// configure the options to stream everything needed for tracking
ST::CaptureSessionSettings settings;
settings.structureCore.depthEnabled = true;
settings.structureCore.visibleEnabled = true; 
settings.structureCore.accelerometerEnabled = true; 
settings.structureCore.gyroscopeEnabled = true;  

// create your delegate class
MyCaptureSessionDelegate delegate;

// attach the delegate to a CaptureSession
ST::CaptureSession captureSession;
captureSession.setDelegate(&delegate);  

// apply settings and start streaming!
captureSession.startMonitoring(settings);
captureSession.startStreaming();

Download

Structure SDK (Cross-Platform) 0.6 Download

Full Release Notes

This release requires Structure Core firmware version 0.9.2. Note that 0.8.x sensors are incompatible and not field upgradeable. If you are not running 0.9.2 or newer, please contact Structure immediately.

Release type: Windows, Linux, Android

Firmware: This SDK requires Structure Core Firmware 0.9.2

Supported Platforms: Windows, Android, Linux

Architectures: x86_64 (Windows/Linux), arm64 (Linux/Android)

  • Structure Core is here! Welcome to all the new customers using the brand new enclosed Structure Core. This release is the first to support the new model ST02D (enclosed) Structure Core.
  • Multi-Sensor Support In this release, you can enumerate and stream from multiple Structure Cores to a single computer (but only one per executable). This is Linux-only for now. Other platforms will follow. Access this feature via these new APIs:
    • Added ST::enumerateConnectedSensors which returns a list of all connected Structure Core devices.
    • Added the sensorSerial field to Structure Core settings in ST::CaptureSessionSettings which allows specifying the serial number of a sensor to use. Other connected sensors are ignored when this option is set.
Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.