On-Device Testing

While Bixby Home Studio (BHS) includes a Try it out feature that lets you test your action flow on a device, it also enables you to do on-device testing of metadata on compatible mobile devices.

On-device testing lets you say user utterances to a Bixby-enabled device to test the metadata of your devices. For example, you can load the metadata to your mobile phone for your lightbulb, and then say "Hi Bixby, turn on my light" to check if your lightbulb turns on. Currently, you can only test utterances through mobile devices. The device used in this guide is a simulated light bulb.

To perform on-device testing, you'll need to set up the following:

Prepare for On-Device Testing

To enable on-device testing, you must first register your device with SmartThings. If you haven't completed this step, create a Samsung Account first. This is the same account you'll use to log into BHS. For more information about registering your device with SmartThings, see the Start Developing guide on the SmartThings Developers website.

Then you need to register the metadata through BHS. You can create new metadata or update existing metadata. Once you upload the new or updated metadata, the first metadata state should be Private.

For more information about registering metadata through BHS, see the Publishing the Bixby Device Graph (BHS Metadata Submission) guide.

You also need to confirm that the MNID and VID of the test device, the MNID and VID of the metadata, the MNID and VID of the Bixby app developer settings, and the MNID of your SmartThings Developers user account all match, as shown in the table below. In addition, the metadata version and metadata state of the Bixby app developer settings and of the metadata must match.

#SmartThings User AccountTest DeviceMetadataBixby App Developer Settings
MNIDmust match the othersmust match the othersmust match the othersmust match the others
VIDN/Amust match the othersmust match the othersmust match the others
Metadata VersionN/AN/Amust match the othersmust match the others
Metadata StateN/AN/Amust match the othersmust match the others

If the MNID of the test device is not visible in the BHS Device tab, as shown below, your account does not have permission to use the device's MNID. You need to request permission from the SmartThings device developer to use the MNID.

Enable On-Device Testing

In order to perform on-device testing, you need to enable the Developer Options button in the Bixby app's settings. This is a hidden feature. To enable this button, first open the Bixby app and tap the More menu at the top right.

A menu with multiple options will appear. Tap the Settings option.

The Bixby settings page will appear. At the bottom, tap About Bixby.

At the top, under "Bixby," you will find the version number. Tap this 5-7 times.

If successfully activated, you will see a notification saying "Developer options activated."

Note

Currently, on-device testing is limited to the mobile device you are using.

You can also import push notifications to BHS for on-device testing.

Enable Push Notifications

  1. In BHS, open the console by clicking on the console icon in the bottom left corner.
  2. Click on Notifications to open the Notifications tab.
  3. Click the Enable button underneath the Notifications Disabled message in the center of the Notifications tab.
  4. A pop-up appears on the left that asks permission to show notifications. Click ALLOW.

You will see a Disable Notifications and a Clear Notifications icon appear in the bottom right corner.

Note

If using a Mac, use the Chrome or Firefox browser to enable and view push notifications.

Set Up Push Notifications on Mobile Devices

  1. On your mobile device, log in to SmartThings with the same Samsung Account you used for BHS when enabling push notifications.
  2. Enable Developer Options and Bixby Home Platform Testing

Send Push Notifications

  1. On your mobile device, give Bixby a command, such as "Hi, Bixby. Turn on the light."
  2. Afterwards, you should receive a notification of the execution result in BHS.

Manage Execution Results

You can load or copy execution results in the Notifications tab.

  1. Hover over a notification with your cursor, which opens a menu to the right.
  2. Make a selection:
    • To load and view the execution graph, click "Load test execution result."
    • To view the execution result in the Log tab, click "Copy execution result to Log".

You can [pop the console out of the main editor window]($TOPICS/bhs-dev-guide/developers/bhs-basics#viewing-console-separately-with-pop-out-console window) into a separate window, to more easily read the console information while working on your metadata graphs.

Also, you can see a notifications indicator in the status bar.

Test Metadata

Now that you've successfully enabled on-device testing, you can test your metadata. After activating developer options, go back to the Bixby settings screen and tap the now-visible Developer options.

On the Developer options screen, tap Bixby Home platform testing. The On-device testing option only applies to developers who are testing Bixby capsules.

The Bixby Home platform testing screen will appear. To enable testing, turn on "Test mode" with the toggle button.

At this point, you will have the option to select metadata (MNID, State, VID, Version) on your mobile device.

When you tap a metadata category, such as MNID or Version, a new screen appears with multiple radio button options. Select the desired option and then tap Select.

Once you've made your selections, you can start testing the metadata. To start your on-device testing, you can return to the Bixby home screen and test utterances for your metadata.

Get Execution Results from Push Notifications

You can import push notifications to Bixby Home Studio (BHS) for on-device testing. With this feature, you can get notifications in BHS when testing Bixby voice commands on mobile devices. This enables you to view and load execution results in BHS for debugging, which includes additional information to help you narrow down any issues, such as the execution path taken during on-device testing, parameters, etc.

To debug execution errors, you can click Load test execution result to show the graph execution result in BHS, which includes information such as parameters, node values, and execution paths from the on-device test.

Click "Copy execution result to Log" to see more information about the error in the Log tab.

See details about failed execution in Log Tab

Test Virtual Devices

If you don't have an actual device to test on, you can create a virtual device. There are two kinds of virtual devices:

  • Virtual devices created within Bixby Home Studio. These support fewer capabilities, but allow you to perform on-device testing.
  • Virtual devices created within SmartThings. These support more capabilities, but do not allow on-device testing.

To create a virtual device within BHS, take the following steps:

  1. Check Use virtual device management in the BHS settings screen.
  2. Click the BHS Device tab. With virtual device management enabled, configurations for virtual devices and virtual device profiles will be visible.
  3. Create a new virtual device profile if one does not exist. Your account must be configured with a valid manufacturer ID (MNID) to create a new profile.
  4. Create a virtual device with an existing virtual device profile.

To create a virtual device within SmartThings, see the SmartThings Community FAQ Creating Virtual Devices with the new architecture.

Note

To go back to using Bixby regularly, you need to disable Bixby Home platform testing.

Video Tutorial: Simulated Devices

The following is a video tutorial on how to set up a simulated device for BHS.