Structure SDK (iOS) Release Notes
- Structure SDK 1.2.1 for iOS
- Structure SDK 1.2 for iOS
- Structure SDK 1.1 for iOS
- Structure SDK 1.0.1 for iOS
- Structure SDK 1.0 for iOS
- Structure SDK 0.12 for iOS
- Structure SDK 0.11.2 for iOS
- Structure SDK 0.11.1 for iOS
- Structure SDK 0.11 for iOS
- Structure SDK 0.10.3 for iOS
- Structure SDK 0.10.2 for iOS
- Structure SDK 0.10.1 for iOS
Structure SDK 1.2.1 for iOS
Released Oct 12, 2021
Our developer community spoke and we listened: by popular demand, we are bringing the iOS Simulator back to the Structure SDK and launching Structure SDK (iOS) 1.2.1. That is the main change in this patch release, but stay tuned: our engineers are working on new features for an upcoming major release!
Structure SDK 1.2 for iOS
Released Jul 13, 2021
We’re excited to announce the launch of Structure SDK (iOS) 1.2! This is our biggest release in a while with connectivity improvements for Structure Sensor (Mark II), removal of
STSensorController support in the SDK, and more! Additionally, firmware 1.2 will be available for download in the Structure app soon, significantly improving support for USB-C iPads. If you'd like to test firmware 1.2, you can download Core Firmware Updater here.
- the Scanner sample app has been updated with features outlined in the new functionality section below.
STSensorControlleris removed in SDK 1.2.
- Calibrator support for iPad Pro 11" (2021) has been added. Support for the iPad Pro 12.9" (2021) along with a new bracket coming soon.
- Depth window adjustment has been added -
kSTCaptureSessionOptionDepthSearchWindowKeyspecifies the size of the search window for depth frame generation. A larger depth window gives you better coverage while smaller windows sizes allow for the resolution of finer details. Occipital recommends a depth value of 11x11. Please note, this function is for advanced users only and can degrade performance if used incorrectly.
// provide to STCaptureSession::startMonitoringWithOptions
kSTCaptureSessionOptionDepthSearchWindowKey: @[@9, @9],
In this example, there is an NSArray of two values,
@[@9, @9], specifying a depth window size of 9x9.
- PLY export (Polygon File format or the Stanford Triangle Format) has been added -
STMesh::writeToFileshould be provided with
kSTMeshWriteOptionFileFormatKey: @(STMeshWriteOptionFileFormatPlyFile). Please note, the output mesh will be in binary format and will be exported in meters. If you need to output in millimeters, a workaround is to scale your vertices in
STMeshprior to calling
- Streaming of high-resolution depth frames (1280x960) has been added.
- A Dark Object Preset has been added -
STCaptureSessionPresetDarkObjectScanning. This preset improves depth coverage in scenes containing dark, shiny objects (i.e. computer monitors, luggage).
- BETA - a depth refinement function has been added -
STDepthFrame::applyDepthRefinementimproves the accuracy of depth data for the upcoming Structure Sensor Pro (ST02B). Specifically, this function removes outliers and neighboring depth values from invalid or bad pixels. Please note, depth refinement is not supported on Structure Sensor (Mark II) (ST02A) or the original Structure Sensor (ST01).
- MTL file on OBJ export is no longer broken
- CoreMotion is now stopped when not streaming
- CoreBluetooth is no longer required when building / linking SDK
STCaptureSessionnow shows correct sensor information on first boot (Structure App now displays serial number correctly)
- Changing streams using the original Structure Sensor no longer causes the Structure App to crash
- iOS Simulators have been removed
- STSensorController has been removed
Firmware 1.2 release notes:
- Improved connectivity for Structure Sensor (Mark II) (ST02A) and Structure Sensor Pro (ST02B).
- Resolved an issue where Structure Sensor (Mark II) was unable to communicate with certain iPad models due to a change by Apple to USB-C authentication.
- Added a new Dark Object Preset that enables improved depth in scenes containing dark, shiny objects (i.e. computer monitors, luggage).
Structure SDK 1.1 for iOS
Released Dec 14, 2020
We are excited to announce that we’ve completed the development and validation of Structure SDK 1.1 for iOS. This release introduces a beta release of our brand-new, Occipital-managed SLAM API, STSLAMManager, for scanning objects with improved accuracy.
NOTE: STSLAMManger is optimized for more powerful iOS devices like the new iPad Air 4 or recent 11" and 12.9" iPad Pros. While we attempt to support as many iOS host devices as possible with Structure SDK, STSLAMManager performs best on newer iPads and currently only supports Structure Sensor Mark II (ST02A) with third-party sensor support coming in a future release.
Introducing Structure SDK for Business
Today we’re announcing our upcoming Structure SDK for Business program, launching in early 2021. We have some exciting features coming including enhanced textures, face-optimized scanning settings and sample code, and Occipital-managed SLAM support for third-party sensors like the Apple TrueDepth camera using STSLAMManager.
- Introducing STSLAMManager into beta. This beta feature includes improvements to object scanning scale accuracy.
- Resolved crashes related to External Accessory Framework (stack traces mentioned “EAInput”, “EASession”, etc)
- Bounding box disappearing when certain presets are selected
- Improved Hybrid (beta) scanning preset, extending the operating range of the sensor
- Fixed a bug with initCameraPose not being applied correctly
- We now stop CoreMotion events when the app backgrounds, saving battery and memory usage
- iPad Air 2 support has been deprecated
- This is the last release that includes STSensorController, all future versions of Structure SDK (iOS) will no longer include this interface
- iOS Host Device Speed: We found that in cases where older, less powerful devices were used some devices saw an increase of tracking slip or stuttering that impacted the final mesh/size accuracy. Our goal is always to support as many iOS devices as possible, however if you are seeing increases in these types of issues you may wish to troubleshoot with a more powerful device
Structure SDK 1.0.1 for iOS
Released Sept 29, 2020
We are excited to announce that the team has completed development and validation of Structure SDK 1.0.1 (iOS). iOS 14 arrived a little sooner than we expected, thus we focused this release solely on ensuring developers and users can update their apps to work on iOS 14. We have many exciting updates coming in a future release, stay tuned!
The following apps have been approved by Apple with these fixes:
- Structure Uplink
- Structure Sensor Calibrator
- Scanner - Structure SDK Sample
- Room Capture - Structure SDK Sample
Full list of changes in SDK 1.0.1:
- Fixed an issue found in iOS 14
Technical Note: We found that Apple/LLVM made changes to
std::chrono::steady_clock at a low level in LLVM 10 (released with iOS 14), which was upgraded from LLVM 9 in iOS 13. This resulted in an issue which appeared as black, white or frozen camera previews in previous versions of Structure SDK when using Structure Sensor Mark II, as frame synchronization relied on the previous behavior.
Structure SDK 1.0 for iOS
Released Jul 23, 2020
We are excited to announce that the team has completed development and validation of Structure SDK 1.0 for iOS. As announced earlier this year, this release adds support for third-party sensors starting with Apple’s new iPad Pro LiDAR Scanner. Future versions of Structure SDK (iOS) will support additional sensors including Apple’s TrueDepth camera.
We’ve found the Apple iPad Pro LiDAR scanner works best for scanning large spaces so we’ve added support to our Room Scanner sample. If you would like to see the sensor in action, along with some special Occipital magic, please download Canvas here 25.
We would like to thank Christopher Worley ( @n6xej) for assisting the team in updating the Room Capture sample app to support LiDAR scanning.
Currently, due to hardware limitations of Apple’s LiDAR Scanner on iPad Pro (50x less resolution vs Structure Sensor Mark II), we do not recommend using Apple LiDAR for object scanning.
Earlier this year, iOS 13.4 launched with some basic APIs for interfacing with Apple’s iPad Pro LiDAR Scanner. When iOS 14 launches out of beta later this year, we’re planning on doing even more with the sensor in Structure SDK (iOS).
In addition to 3rd party sensor support, we have been busy making improvements to Structure SDK (iOS)!
Full list of changes in SDK 1.0:
- Reverted STMeshWriteOptionFileFormatObjFile naming so that .jpg files are named Model.jpg rather than Model0.jpg
- kSTCaptureSessionOptionDepthSensorEnabledKey was “NO” in documentation, and “YES” in the code by default. We have aligned both of these sources to “YES”
- Updated sample app icons to reflect new Structure branding
- Revised battery logic related to STCaptureSessionSensorModeBatteryDepleted to trigger depleted status at 0% rather than 10% for connected/initialized sensors
- kSTMapperVolumeCutBelowSupportPlaneKey was not being respected when legacy mapper was used. This is intentional, we have updated error messaging and documentation to reflect this non compatibility.
- Reflecting the continued deprecation of STSensorController, we have removed the “low level” SDK examples that used STSensorController. STSensorController is still available in the SDK, but we highly suggest new and existing apps move to STCaptureSensorController before STSensorController is fully removed from the SDK. We are seeking community feedback regarding this change.
- (BETA) We have added a new depth preset Hybrid mode
- (BETA) We have added a new SDK connection to allow recording in QVGA depth mode
Structure SDK 0.12 for iOS
Released May 19, 2020
Today we are excited to announce the newest version of Structure SDK (iOS) 0.12! Our teams have been focused on improving performance, accuracy and stability for this build.
Structure SDK (iOS) 0.12 is highly encouraged to be paired with a Structure Sensor (Mark II) running Firmware 1.0, or for Structure Sensor (Original) devices Firmware 2.0.
- Introducing support for Firmware version 1.0
- Fixed miscellaneous crashing related to background and foreground app transitions
- Fixed miscellaneous crashing and failures related to starting and stopping streaming
- Fixed a crash related to colorCameraPropertiesQueue
- Fixed an issue with some ST02A sensors experiencing dropping frame rates after a period of time
- Battery level reporting has been slightly changed. When a sensor is in low power mode, and is unable to start streaming STCaptureSessionSensorModeBatteryDepleted will be emitted.
- STCaptureSessionSensorModeBatteryDepleted events will now be triggered on the main thread, allowing UI updates to take place.
Structure SDK 0.11.2 for iOS
Released Jan 7, 2020
Today (7th of January, 2020) we have released Structure SDK 0.11.2.
This is a minor update with some quick but important fixes. It still includes the major updates seen in SDK 0.11 but additionally fixes a few last-minute bugs we found. For a full post regarding the updates in SDK 0.11, and 0.11.1 please check out the below post.
Structure SDK 0.11.2 Release Notes
- Fixed some bugs where Structure Sensor Mark II would either fail to stream after restarting streams several times, or the frame rate would slow down every time streaming would restart.
Structure SDK 0.11.1 for iOS
Released Dec 18, 2019
Structure SDK 0.11.1 Release Notes
- Fixed a bug in our SLAM code that affects room level scanning when using our wide-vision lens attachment
- Fixed a bug in the docs for launchStructureAppOrGoToAppStore. We specified in the docs that you should add structure to LSApplicationQueriesSchemes in the Info.plist, however this should have been structure-app.
Structure SDK 0.11 for iOS
Released Dec 9, 2019
This is a major update to Structure SDK and it comes with a host of stability updates and improvements for Structure Sensor (Mark II).
This SDK is being released in parallel with a new Structure App and Mark II firmware update. For best results please use the latest SDK along with the latest firmware. Users can update the firmware by downloading the Structure app and running the included firmware updater. The firmware update is not available directly through the Structure SDK (for iOS) 0.11.
For any developers who may have run into issues with App Store rejection due to obfuscated code, please see our App Store Obfuscation Rejection Update forum post.
Structure SDK 0.11 Release Notes:
- Structure SDK headers have been significantly refactored and now include a module map. These changes improve clarity, and will make Swift porting simpler. Now, instead of including separate classes, you only need to include the umbrella header ‘Structure.h’.
- Driver updates to improve Structure Sensor (Mark II) connection and stability.
- STSensorController has been deprecated. Verbose logs warning that STSensorController is no longer supported have been added. All apps built on STSensorController should migrate to STCaptureSession (details here: Announcing Structure SDK (iOS) 0.9: STCaptureSession). All samples built on STSensorController will still work for the time being, but will not be under continued development.
- We have added a sample notification in the Scanner Sample prompting users to update to the latest firmware. The following code will prompt users to download the Structure App from the app store to update to the latest firmware version:
bool needToUpgradeFirmware = userInstructions & STCaptureSessionUserInstructionFirmwareUpdateRequired;
bool needToUpgradeFirmware = [[STSensorController sharedController] isFirmwareUpdateNeeded];
- Fixed a bug where apps would crash when backgrounded by the OS. This also resolves a crash that would happen when the camera permissions dialog box appears for the first time.
- Fixed a bug where Structure Sensor Mark II would not reconnect after an app was backgrounded
- Fixed a bug where unplugging and replugging Structure Sensor (Mark II) would crash the application
- “Improved Tracker” is not currently optimized for Structure Sensor Mark II. Optimization for the “Improved Tracker” will be added in the near future. In Structure SDK (for iOS) 0.11 “Legacy Tracker” has been set to the default tracking mode in the Scanner sample application. This option can be toggled by by setting kSTTrackerLegacyKey to YES in the STTracker options, or using the gear icon in the upper left corner of the Scanner Sample app.
- On iOS 13 sensors may not be detected on app initialization. The sensor must be unplugged and replugged to restore connection. This behavior does not exhibit itself on iOS 12.
- kSTMapperVolumeCutBelowSupportPlaneKey clips the ground plane even when set to
NOwhen using the legacy mapper.
- We’ve seen intermittent framerate reduction when repeatedly stopping and restarting streaming on Mark II. Triggers include changing depth modes or returning to scan mode after saving a mesh. If you encounter reduced framerate, please quit and restart your app.
Structure SDK 0.10.3 for iOS
Released Oct 23, 2019
Today we are releasing Structure SDK (for iOS) 0.10.3. This release includes iOS 13 compatibility updates, fixes a IMU related tracking regression, and resolves a number of minor bugs.
Please note – if you’re upgrading from an earlier version (0.10 or earlier) be sure to read the 0.10.1 notes on the HERE. Applications built on 0.10.1+ require special steps when submitting to the app store.
REMOVED setupIMU METHOD
Since SDK 0.9, the STCaptureSession provides direct access to IMU data. In the Room Capture and Scanner sample apps, we have been using the old setupIMU initialization for IMU events, while also utilizing the IMU information from STCaptureSession. This caused regressions in tracking, especially on iOS / iPadOS 13. We have removed the setupIMU method and all corresponding calls, which should improve performance of the sample apps.
SCAN BUTTON NOT WORKING IN SCANNER SAMPLE
Structure SDK 0.10.3 fixes an error in the Scanner sample app in which scanning would not start when the Scan button is pressed.
CALLING sensorBatteryLevel AND getBatteryChargePercentage WHEN SENSOR IS DISCONNECTED NO LONGER CRASHES APP
We have fixed a crash related to calling STCaptureSession:sensorBatteryLevel or STSensorController:getBatteryChargePercentage after a Structure Sensor Mark II is disconnected. Please contact support if you continue to experience issues pertaining to the use of these APIs.
Structure SDK 0.10.2 for iOS
Released Dec 9, 2019
Today we are releasing Structure SDK (for iOS) 0.10.2. This update fixes an issue where Mark II units would incorrectly report that the battery needed charging at 60% charge, as well as including stability updates to reduce situations where the applications would freeze on sensor disconnect.
Please note – this update is a targeted patch on top of the Structure SDK (for iOS) 0.10.1 release. If you’re upgrading from an earlier version (0.10 or earlier) be sure to read the 0.10.1 notes on the HERE 45. Special steps are required for apps built on 0.10.1 and above when submitting to the app store.
MARK II BATTERY CHARGE NOW CORRECTLY REPORTING
Structure SDK (for iOS) 0.10.2 now accurate handles battery state checking on Structure Sensor Mark II. The SDK no longer incorrectly reports that the battery needs charging when your Mark II battery is at 60% or below. This update will significantly extend the battery life on Mark II sensors.
APP FREEZING DUE TO sensorDidDisconnect BUG RESOLVED
This update resolves an issue where
sensorDidDisconnect was not being properly signalled in both the
STCaptureSession with Structure Sensor Mark II. This bug would cause applications to freeze or crash when the sensor disconnected. Now apps will be properly signalled when a Structure Sensor Mark II device disconnects from the host iOS device.
STDepthFrame API CHANGES
Structure SDK (for iOS) 0.10.2 adds a new API to
STSensorController. While these methods will be supported for the time being (as they are required in configurations that use hardware registered depth, which has been deprecated for some time), in future versions of the SDK we plan to remove them along with hardware registered depth configurations. We hope this new API helps developers by making the current color from depth extrinsics clearer.
REMOVED CoreBluetooth AS A DEPENDENCY
Certain apps built on 0.10.1 were rejected during the App Store submission process due to
Missing Purpose String in Info.plist. We have removed CoreBluetooth as a dependency, and apps build on 0.10.2 will no longer trigger auto trigger rejections from the App Store for this reason.
Structure SDK 0.10.1 for iOS
Released Sept 30, 2019
Today we are pleased to announce the release of Structure SDK (for iOS) 0.10.1. This update includes two primary updates as well as a number of improvements and changes across the board. First, the key updates:
APP STORE SUBMISSION
Our last release triggered an automated flag in Apple’s process regarding an obfuscated symbols error when submitting an app to App Store Connect. We have addressed this with the Structure SDK 0.10.1 release, and it should be easier for developers to submit apps now.
All apps built on Structure SDK 0.10 should be updated to 0.10.1 before submission to the App Store.
The Structure SDK 0.10 release included many symbols from some of Occipital’s internal dependencies, such as Eigen3 and OpenCV. This could result in symbol clashes that made it difficult for our developers to include their own versions of these libraries. To prevent this from happening in the future, we have begun providing a dynamic framework for Structure SDK releases.
In order to use Structure SDK 0.10.1, you will need to embed the Structure.framework into your app on the app target’s “General” page in Xcode.
A tutorial for what to do can be found HERE. Please read this post, as there are additional instructions for those who want to submit their apps to the App Store.
The Structure SDK 0.10.1 release also includes the following updates:
New STTracker Methods
We added two new methods to STTracker (updateCameraPoseWithGyro and updateCameraPoseWithAccel) so that developers can pass raw Accelerometer and Gyroscope events from the iOS IMU to our tracker. The ability to turn off CoreMotion in favor of raw IMU events was added along with STCaptureSession in SDK 0.9, but we didn’t add the corresponding tracker methods at that time. Those methods are now available.
The Sample App projects have been updated to be backwards compatible with XCode 10.0 or newer
In addition, many recommended settings have been enabled on the projects according to modern XCode standards and Apple recommendations.
C functions are now exposed as C symbols, rather than C++ symbols
We hope this helps makes things easier for developers who write Swift wrappers for the Structure SDK
HAS_LIBCXX has been added to the project preprocessor definitions rather than defined in the Sample App View Controllers
We hope this also makes it easier for developers using our Sample Apps as a base for their own projects.
The Structure SDK 0.10.1 includes the following bug fixes:
- Fixed a bug when writing OCCs with high-resolution color frames in STCaptureSession
- Fixed a bug where the firmwareRevision and hardwareRevision properties in STSensorController returned nil when a Mark II device is plugged in
- Fixed how we display the mesh view in Scanner and RoomCapture on iPadOS 13
You may observe the message “please put the model back in view” in Scanner right after starting a scan. If you observe this happening more often than what happens with the original Structure Sensor, please contact us at firstname.lastname@example.org