Structure SDK (iOS) Release Notes
- Structure SDK 2.1 for iOS
- Structure SDK 2.0.1 for iOS
- Structure SDK 2.0 for iOS
- Structure SDK 1.2.1 for iOS
- Structure SDK 1.2 for iOS
- Structure SDK 1.1 for iOS
- Structure SDK 1.0.1 for iOS
- Structure SDK 1.0 for iOS
- Structure SDK 0.12 for iOS
- Structure SDK 0.11.2 for iOS
- Structure SDK 0.11.1 for iOS
- Structure SDK 0.11 for iOS
- Structure SDK 0.10.3 for iOS
- Structure SDK 0.10.2 for iOS
- Structure SDK 0.10.1 for iOS
Structure SDK 2.1 for iOS
Released Aug 17, 2022
We are excited to announce the launch of Structure SDK (iOS) 2.1! This is our first attempt to adopt a tick/tock major/maintenance release cadence, a practice we plan to continue to follow in the future.
Structure (iOS) SDK 2.1 is now provided as a XCFramework. This simplifies integration with Swift applications and enables other architectures for iOS Simulator. You can see more at Distributing Binary Frameworks as Swift Packages.
We’re also releasing a new Beta of STCaptureSession. We recommend that all developers try this prior to the end of 2022 to ensure that their apps will be compatible when it is publicly released with Structure SDK 3.0 for iOS.
Please Note: We will be sunsetting original Structure Sensor (model: ST01) support with the release of Structure SDK 3.0 for iOS. ST01 may work with future releases of our SDK but new features will not be tested or verified with the deprecated sensor. ST01 will not be configured to work with newer iPads/iPhones.
- iOS Simulator support added for arm64 architecture. This enables running simulator on Apple Silicon (M1).
- An option to specify resolution of IR frames. Note: the highest resolution is limited to a maximum frame rate of 10fps.
- An option to specify if rectification of IR frames is required (true by default).
- The ability to turn the IR projector ON and OFF.
- A read only property for STInfraredFrame to read frame exposure duration
- An option to specify ARKit configuration (required in case of using TrueDepth camera)
- A read only property to query IR camera extrinsics taking into account the rectification
- New functions to query mesh during scanning with STSLAMManager
- GLES-based visualization has been deprecated by Apple and will be removed in future releases. Please use Metal-based visualization API provided in the Samples/Shared folder.
- An old option for ARKit (replaced with
Structure SDK 2.0.1 for iOS
Released Apr 12, 2022
After receiving notification from fellow developers that they are having difficulties obtaining approval from Apple for releasing their applications while using Structure SDK 2.0, including True Depth Support, we decided to deploy a quick fix, making available a patch release (Structure SDK 2.0.1) including two flavors of the SDK: with and without True Depth support. Additionally, SDK stability has been improved.
If your application requires use of Apple’s True Depth sensor, please use the framework available in the folder: Frameworks/TrueDepth/. Also, be prepared to answer questions from Apple’s review team regarding the use of True Depth sensor by your application.
Otherwise, please use the framework available in the Frameworks/ folder, as usual.
Thank you for your understanding.
Structure SDK 2.0 for iOS
Released Mar 9, 2022
Today we are announcing Structure SDK 2.0 with beta support for the Apple TrueDepth camera found on many modern iOS devices. This release brings Occipital’s best-in-class 3D scanning and reconstruction available to hundreds of millions of devices, allowing Structure SDK developers to easily augment their existing apps to enable scanning using this widely accessible sensor. That feature is free to try, and our commercial users have a choice of either buying the Structure Sensor - the leading sensor used for healthcare applications - and using the SDK for free, as they always have, or licensing the SDK for use with True Depth sensors.
This release also includes a new sample app: 3DFaceScan, available to pre-qualified developers. This is our first sample app using Swift, and it showcases the use of Structure SDK 2.0 to perform face scans using the Apple TrueDepth camera. For the initial release, we’re selecting a limited number of partners into the program. The 3DFaceScan app will be available on Apple’s App Store with a 7-day free trial period in the coming days.
- Apple TrueDepth camera now supported in Structure SDK (iOS)
- 3DFaceScan app available using Apple’s TrueDepth sensor
- Battery level indicator included in sample apps
- Texturing algorithm added - ideal for rooms and buildings (Room Capture)
- Ability to apply mesh transform (ie. Scale from meters to millimeters or applying transformation matrix)
- Ability to mirror/Rotate the Mesh Function (i.e. Portrait / Landscape)
- Hole-filling method (Liepa, in addition to Poisson)
- Bounding box mirroring/rotation and texture renderer for different screen orientations functions added to Structure SDK (iOS)
- Detailed tutorial on creating a new project (under Reference)
- Additional Structure Sensor Pro Preset: Medium (0.5-3m range)
- New sample application using Swift (3DFaceScan sample code available to select developers)
- Settings User Interface has been cleaned up to be more user-friendly
- Expanded Sharing Options (e.g. AirDrop, To Device, Email) added to Scanner sample app
- Export File Formats
- Bounding Box Options
- Fixed Position
- Align to Gravity
- Firmware Update (Available but not required for patch versions)
- Customize Depth Rendering Color (Color feedback to user during bounding box placement)
- Mesh Rendering Function (ie. Red means too close and not in the scanning area)
- Minimum tracking distance specifications (ie. 10 cm for face scanning)
- Fixed incorrect metadata attached to OBJ files
**Please see the change log in the index.html for specific function names and examples.**
Structure SDK 1.2.1 for iOS
Released Oct 12, 2021
Our developer community spoke and we listened: by popular demand, we are bringing the iOS Simulator back to the Structure SDK and launching Structure SDK (iOS) 1.2.1. That is the main change in this patch release, but stay tuned: our engineers are working on new features for an upcoming major release!
Structure SDK 1.2 for iOS
Released Jul 13, 2021
We’re excited to announce the launch of Structure SDK (iOS) 1.2! This is our biggest release in a while with connectivity improvements for Structure Sensor (Mark II), removal of
STSensorController support in the SDK, and more! Additionally, firmware 1.2 will be available for download in the Structure app soon, significantly improving support for USB-C iPads. If you'd like to test firmware 1.2, you can download Core Firmware Updater here.
- the Scanner sample app has been updated with features outlined in the new functionality section below.
STSensorControlleris removed in SDK 1.2.
- Calibrator support for iPad Pro 11" (2021) has been added. Support for the iPad Pro 12.9" (2021) along with a new bracket coming soon.
- Depth window adjustment has been added -
kSTCaptureSessionOptionDepthSearchWindowKeyspecifies the size of the search window for depth frame generation. A larger depth window gives you better coverage while smaller windows sizes allow for the resolution of finer details. Occipital recommends a depth value of 11x11. Please note, this function is for advanced users only and can degrade performance if used incorrectly.
// provide to STCaptureSession::startMonitoringWithOptions
kSTCaptureSessionOptionDepthSearchWindowKey: @[@9, @9],
In this example, there is an NSArray of two values,
@[@9, @9], specifying a depth window size of 9x9.
- PLY export (Polygon File format or the Stanford Triangle Format) has been added -
STMesh::writeToFileshould be provided with
kSTMeshWriteOptionFileFormatKey: @(STMeshWriteOptionFileFormatPlyFile). Please note, the output mesh will be in binary format and will be exported in meters. If you need to output in millimeters, a workaround is to scale your vertices in
STMeshprior to calling
- Streaming of high-resolution depth frames (1280x960) has been added.
- A Dark Object Preset has been added -
STCaptureSessionPresetDarkObjectScanning. This preset improves depth coverage in scenes containing dark, shiny objects (i.e. computer monitors, luggage).
- BETA - a depth refinement function has been added -
STDepthFrame::applyDepthRefinementimproves the accuracy of depth data for the upcoming Structure Sensor Pro (ST02B). Specifically, this function removes outliers and neighboring depth values from invalid or bad pixels. Please note, depth refinement is not supported on Structure Sensor (Mark II) (ST02A) or the original Structure Sensor (ST01).
- MTL file on OBJ export is no longer broken
- CoreMotion is now stopped when not streaming
- CoreBluetooth is no longer required when building / linking SDK
STCaptureSessionnow shows correct sensor information on first boot (Structure App now displays serial number correctly)
- Changing streams using the original Structure Sensor no longer causes the Structure App to crash
- iOS Simulators have been removed
- STSensorController has been removed
Firmware 1.2 release notes:
- Improved connectivity for Structure Sensor (Mark II) (ST02A) and Structure Sensor Pro (ST02B).
- Resolved an issue where Structure Sensor (Mark II) was unable to communicate with certain iPad models due to a change by Apple to USB-C authentication.
- Added a new Dark Object Preset that enables improved depth in scenes containing dark, shiny objects (i.e. computer monitors, luggage).
Structure SDK 1.1 for iOS
Released Dec 14, 2020
We are excited to announce that we’ve completed the development and validation of Structure SDK 1.1 for iOS. This release introduces a beta release of our brand-new, Occipital-managed SLAM API, STSLAMManager, for scanning objects with improved accuracy.
NOTE: STSLAMManger is optimized for more powerful iOS devices like the new iPad Air 4 or recent 11" and 12.9" iPad Pros. While we attempt to support as many iOS host devices as possible with Structure SDK, STSLAMManager performs best on newer iPads and currently only supports Structure Sensor Mark II (ST02A) with third-party sensor support coming in a future release.
Introducing Structure SDK for Business
Today we’re announcing our upcoming Structure SDK for Business program, launching in early 2021. We have some exciting features coming including enhanced textures, face-optimized scanning settings and sample code, and Occipital-managed SLAM support for third-party sensors like the Apple TrueDepth camera using STSLAMManager.
- Introducing STSLAMManager into beta. This beta feature includes improvements to object scanning scale accuracy.
- Resolved crashes related to External Accessory Framework (stack traces mentioned “EAInput”, “EASession”, etc)
- Bounding box disappearing when certain presets are selected
- Improved Hybrid (beta) scanning preset, extending the operating range of the sensor
- Fixed a bug with initCameraPose not being applied correctly
- We now stop CoreMotion events when the app backgrounds, saving battery and memory usage
- iPad Air 2 support has been deprecated
- This is the last release that includes STSensorController, all future versions of Structure SDK (iOS) will no longer include this interface
- iOS Host Device Speed: We found that in cases where older, less powerful devices were used some devices saw an increase of tracking slip or stuttering that impacted the final mesh/size accuracy. Our goal is always to support as many iOS devices as possible, however if you are seeing increases in these types of issues you may wish to troubleshoot with a more powerful device
Structure SDK 1.0.1 for iOS
Released Sept 29, 2020
We are excited to announce that the team has completed development and validation of Structure SDK 1.0.1 (iOS). iOS 14 arrived a little sooner than we expected, thus we focused this release solely on ensuring developers and users can update their apps to work on iOS 14. We have many exciting updates coming in a future release, stay tuned!
The following apps have been approved by Apple with these fixes:
- Structure Uplink
- Structure Sensor Calibrator
- Scanner - Structure SDK Sample
- Room Capture - Structure SDK Sample
Full list of changes in SDK 1.0.1:
- Fixed an issue found in iOS 14
Technical Note: We found that Apple/LLVM made changes to
std::chrono::steady_clock at a low level in LLVM 10 (released with iOS 14), which was upgraded from LLVM 9 in iOS 13. This resulted in an issue which appeared as black, white or frozen camera previews in previous versions of Structure SDK when using Structure Sensor Mark II, as frame synchronization relied on the previous behavior.
Structure SDK 1.0 for iOS
Released Jul 23, 2020
We are excited to announce that the team has completed development and validation of Structure SDK 1.0 for iOS. As announced earlier this year, this release adds support for third-party sensors starting with Apple’s new iPad Pro LiDAR Scanner. Future versions of Structure SDK (iOS) will support additional sensors including Apple’s TrueDepth camera.
We’ve found the Apple iPad Pro LiDAR scanner works best for scanning large spaces so we’ve added support to our Room Scanner sample. If you would like to see the sensor in action, along with some special Occipital magic, please download Canvas here 25.
We would like to thank Christopher Worley ( @n6xej) for assisting the team in updating the Room Capture sample app to support LiDAR scanning.
Currently, due to hardware limitations of Apple’s LiDAR Scanner on iPad Pro (50x less resolution vs Structure Sensor Mark II), we do not recommend using Apple LiDAR for object scanning.
Earlier this year, iOS 13.4 launched with some basic APIs for interfacing with Apple’s iPad Pro LiDAR Scanner. When iOS 14 launches out of beta later this year, we’re planning on doing even more with the sensor in Structure SDK (iOS).
In addition to 3rd party sensor support, we have been busy making improvements to Structure SDK (iOS)!
Full list of changes in SDK 1.0:
- Reverted STMeshWriteOptionFileFormatObjFile naming so that .jpg files are named Model.jpg rather than Model0.jpg
- kSTCaptureSessionOptionDepthSensorEnabledKey was “NO” in documentation, and “YES” in the code by default. We have aligned both of these sources to “YES”
- Updated sample app icons to reflect new Structure branding
- Revised battery logic related to STCaptureSessionSensorModeBatteryDepleted to trigger depleted status at 0% rather than 10% for connected/initialized sensors
- kSTMapperVolumeCutBelowSupportPlaneKey was not being respected when legacy mapper was used. This is intentional, we have updated error messaging and documentation to reflect this non compatibility.
- Reflecting the continued deprecation of STSensorController, we have removed the “low level” SDK examples that used STSensorController. STSensorController is still available in the SDK, but we highly suggest new and existing apps move to STCaptureSensorController before STSensorController is fully removed from the SDK. We are seeking community feedback regarding this change.
- (BETA) We have added a new depth preset Hybrid mode
- (BETA) We have added a new SDK connection to allow recording in QVGA depth mode
Structure SDK 0.12 for iOS
Released May 19, 2020
Today we are excited to announce the newest version of Structure SDK (iOS) 0.12! Our teams have been focused on improving performance, accuracy and stability for this build.
Structure SDK (iOS) 0.12 is highly encouraged to be paired with a Structure Sensor (Mark II) running Firmware 1.0, or for Structure Sensor (Original) devices Firmware 2.0.
- Introducing support for Firmware version 1.0
- Fixed miscellaneous crashing related to background and foreground app transitions
- Fixed miscellaneous crashing and failures related to starting and stopping streaming
- Fixed a crash related to colorCameraPropertiesQueue
- Fixed an issue with some ST02A sensors experiencing dropping frame rates after a period of time
- Battery level reporting has been slightly changed. When a sensor is in low power mode, and is unable to start streaming STCaptureSessionSensorModeBatteryDepleted will be emitted.
- STCaptureSessionSensorModeBatteryDepleted events will now be triggered on the main thread, allowing UI updates to take place.
Structure SDK 0.11.2 for iOS
Released Jan 7, 2020
Today (7th of January, 2020) we have released Structure SDK 0.11.2.
This is a minor update with some quick but important fixes. It still includes the major updates seen in SDK 0.11 but additionally fixes a few last-minute bugs we found. For a full post regarding the updates in SDK 0.11, and 0.11.1 please check out the below post.
Structure SDK 0.11.2 Release Notes
- Fixed some bugs where Structure Sensor Mark II would either fail to stream after restarting streams several times, or the frame rate would slow down every time streaming would restart.
Structure SDK 0.11.1 for iOS
Released Dec 18, 2019
Structure SDK 0.11.1 Release Notes
- Fixed a bug in our SLAM code that affects room level scanning when using our wide-vision lens attachment
- Fixed a bug in the docs for launchStructureAppOrGoToAppStore. We specified in the docs that you should add structure to LSApplicationQueriesSchemes in the Info.plist, however this should have been structure-app.
Structure SDK 0.11 for iOS
Released Dec 9, 2019
This is a major update to Structure SDK and it comes with a host of stability updates and improvements for Structure Sensor (Mark II).
This SDK is being released in parallel with a new Structure App and Mark II firmware update. For best results please use the latest SDK along with the latest firmware. Users can update the firmware by downloading the Structure app and running the included firmware updater. The firmware update is not available directly through the Structure SDK (for iOS) 0.11.
For any developers who may have run into issues with App Store rejection due to obfuscated code, please see our App Store Obfuscation Rejection Update forum post.
Structure SDK 0.11 Release Notes:
- Structure SDK headers have been significantly refactored and now include a module map. These changes improve clarity, and will make Swift porting simpler. Now, instead of including separate classes, you only need to include the umbrella header ‘Structure.h’.
- Driver updates to improve Structure Sensor (Mark II) connection and stability.
- STSensorController has been deprecated. Verbose logs warning that STSensorController is no longer supported have been added. All apps built on STSensorController should migrate to STCaptureSession (details here: Announcing Structure SDK (iOS) 0.9: STCaptureSession). All samples built on STSensorController will still work for the time being, but will not be under continued development.
- We have added a sample notification in the Scanner Sample prompting users to update to the latest firmware. The following code will prompt users to download the Structure App from the app store to update to the latest firmware version:
bool needToUpgradeFirmware = userInstructions & STCaptureSessionUserInstructionFirmwareUpdateRequired;
bool needToUpgradeFirmware = [[STSensorController sharedController] isFirmwareUpdateNeeded];
- Fixed a bug where apps would crash when backgrounded by the OS. This also resolves a crash that would happen when the camera permissions dialog box appears for the first time.
- Fixed a bug where Structure Sensor Mark II would not reconnect after an app was backgrounded
- Fixed a bug where unplugging and replugging Structure Sensor (Mark II) would crash the application
- “Improved Tracker” is not currently optimized for Structure Sensor Mark II. Optimization for the “Improved Tracker” will be added in the near future. In Structure SDK (for iOS) 0.11 “Legacy Tracker” has been set to the default tracking mode in the Scanner sample application. This option can be toggled by by setting kSTTrackerLegacyKey to YES in the STTracker options, or using the gear icon in the upper left corner of the Scanner Sample app.
- On iOS 13 sensors may not be detected on app initialization. The sensor must be unplugged and replugged to restore connection. This behavior does not exhibit itself on iOS 12.
- kSTMapperVolumeCutBelowSupportPlaneKey clips the ground plane even when set to
NOwhen using the legacy mapper.
- We’ve seen intermittent framerate reduction when repeatedly stopping and restarting streaming on Mark II. Triggers include changing depth modes or returning to scan mode after saving a mesh. If you encounter reduced framerate, please quit and restart your app.
Structure SDK 0.10.3 for iOS
Released Oct 23, 2019
Today we are releasing Structure SDK (for iOS) 0.10.3. This release includes iOS 13 compatibility updates, fixes a IMU related tracking regression, and resolves a number of minor bugs.
Please note – if you’re upgrading from an earlier version (0.10 or earlier) be sure to read the 0.10.1 notes on the HERE. Applications built on 0.10.1+ require special steps when submitting to the app store.
REMOVED setupIMU METHOD
Since SDK 0.9, the STCaptureSession provides direct access to IMU data. In the Room Capture and Scanner sample apps, we have been using the old setupIMU initialization for IMU events, while also utilizing the IMU information from STCaptureSession. This caused regressions in tracking, especially on iOS / iPadOS 13. We have removed the setupIMU method and all corresponding calls, which should improve performance of the sample apps.
SCAN BUTTON NOT WORKING IN SCANNER SAMPLE
Structure SDK 0.10.3 fixes an error in the Scanner sample app in which scanning would not start when the Scan button is pressed.
CALLING sensorBatteryLevel AND getBatteryChargePercentage WHEN SENSOR IS DISCONNECTED NO LONGER CRASHES APP
We have fixed a crash related to calling STCaptureSession:sensorBatteryLevel or STSensorController:getBatteryChargePercentage after a Structure Sensor Mark II is disconnected. Please contact support if you continue to experience issues pertaining to the use of these APIs.
Structure SDK 0.10.2 for iOS
Released Dec 9, 2019
Today we are releasing Structure SDK (for iOS) 0.10.2. This update fixes an issue where Mark II units would incorrectly report that the battery needed charging at 60% charge, as well as including stability updates to reduce situations where the applications would freeze on sensor disconnect.
Please note – this update is a targeted patch on top of the Structure SDK (for iOS) 0.10.1 release. If you’re upgrading from an earlier version (0.10 or earlier) be sure to read the 0.10.1 notes on the HERE 45. Special steps are required for apps built on 0.10.1 and above when submitting to the app store.
MARK II BATTERY CHARGE NOW CORRECTLY REPORTING
Structure SDK (for iOS) 0.10.2 now accurate handles battery state checking on Structure Sensor Mark II. The SDK no longer incorrectly reports that the battery needs charging when your Mark II battery is at 60% or below. This update will significantly extend the battery life on Mark II sensors.
APP FREEZING DUE TO sensorDidDisconnect BUG RESOLVED
This update resolves an issue where
sensorDidDisconnect was not being properly signalled in both the
STCaptureSession with Structure Sensor Mark II. This bug would cause applications to freeze or crash when the sensor disconnected. Now apps will be properly signalled when a Structure Sensor Mark II device disconnects from the host iOS device.
STDepthFrame API CHANGES
Structure SDK (for iOS) 0.10.2 adds a new API to
STSensorController. While these methods will be supported for the time being (as they are required in configurations that use hardware registered depth, which has been deprecated for some time), in future versions of the SDK we plan to remove them along with hardware registered depth configurations. We hope this new API helps developers by making the current color from depth extrinsics clearer.
REMOVED CoreBluetooth AS A DEPENDENCY
Certain apps built on 0.10.1 were rejected during the App Store submission process due to
Missing Purpose String in Info.plist. We have removed CoreBluetooth as a dependency, and apps build on 0.10.2 will no longer trigger auto trigger rejections from the App Store for this reason.
Structure SDK 0.10.1 for iOS
Released Sept 30, 2019
Today we are pleased to announce the release of Structure SDK (for iOS) 0.10.1. This update includes two primary updates as well as a number of improvements and changes across the board. First, the key updates:
APP STORE SUBMISSION
Our last release triggered an automated flag in Apple’s process regarding an obfuscated symbols error when submitting an app to App Store Connect. We have addressed this with the Structure SDK 0.10.1 release, and it should be easier for developers to submit apps now.
All apps built on Structure SDK 0.10 should be updated to 0.10.1 before submission to the App Store.
The Structure SDK 0.10 release included many symbols from some of Occipital’s internal dependencies, such as Eigen3 and OpenCV. This could result in symbol clashes that made it difficult for our developers to include their own versions of these libraries. To prevent this from happening in the future, we have begun providing a dynamic framework for Structure SDK releases.
In order to use Structure SDK 0.10.1, you will need to embed the Structure.framework into your app on the app target’s “General” page in Xcode.
A tutorial for what to do can be found HERE. Please read this post, as there are additional instructions for those who want to submit their apps to the App Store.
The Structure SDK 0.10.1 release also includes the following updates:
New STTracker Methods
We added two new methods to STTracker (updateCameraPoseWithGyro and updateCameraPoseWithAccel) so that developers can pass raw Accelerometer and Gyroscope events from the iOS IMU to our tracker. The ability to turn off CoreMotion in favor of raw IMU events was added along with STCaptureSession in SDK 0.9, but we didn’t add the corresponding tracker methods at that time. Those methods are now available.
The Sample App projects have been updated to be backwards compatible with XCode 10.0 or newer
In addition, many recommended settings have been enabled on the projects according to modern XCode standards and Apple recommendations.
C functions are now exposed as C symbols, rather than C++ symbols
We hope this helps makes things easier for developers who write Swift wrappers for the Structure SDK
HAS_LIBCXX has been added to the project preprocessor definitions rather than defined in the Sample App View Controllers
We hope this also makes it easier for developers using our Sample Apps as a base for their own projects.
The Structure SDK 0.10.1 includes the following bug fixes:
- Fixed a bug when writing OCCs with high-resolution color frames in STCaptureSession
- Fixed a bug where the firmwareRevision and hardwareRevision properties in STSensorController returned nil when a Mark II device is plugged in
- Fixed how we display the mesh view in Scanner and RoomCapture on iPadOS 13
You may observe the message “please put the model back in view” in Scanner right after starting a scan. If you observe this happening more often than what happens with the original Structure Sensor, please contact us at email@example.com