Kyle Van Berendonck

Important changes to forums and questions

All forums and questions are now archived. To start a new conversation or read the latest updates go to forums.mbed.com.

8 years, 3 months ago.

Changing name of test affects whether it passes or fails

Hi,

When I use the example test file, delete my .build folder, delete every other folder in TESTS and run it with -n test-name-here a particular test fails unless its file name is EXACTLY the same as the example.

In this case if I rename the sample-test folder to sample-test2 run

mbed test -n tests-sample-test-group-sample-test2 -t GCC_ARM -m LPC1768

it fails with a TIMEOUT error and doesn't display a list of the tests in the file

If I put the folder name back and run

-n tests-sample-test-group-sample-test -t GCC_ARM -m LPC1768

it succeeds in showing the two tests, OK and FAIL, but more importantly it actually shows a list of the tests in the file.

Consider below terminal output shasum of build file is same, there is no logical reason why anything different should happen

Kyles-MacBook-Pro:sample-test-group kvanb$ shasum sample-test/sample-test.bin
63aed24d0793403122a82310ab179b17cc64461b  sample-test/sample-test.bin
Kyles-MacBook-Pro:sample-test-group kvanb$ shasum sample-test2/sample-test2.bin
63aed24d0793403122a82310ab179b17cc64461b  sample-test2/sample-test2.bin

I have power cycled the device multiple times

Full output of fail

> mbedgt: greentea test automation tool ver. 1.2.0
> mbedgt: test specification file '/Users/kvanb/git/ams-test-cyclic/.build/tests/LPC1768/GCC_ARM/test_spec.json' (specified with > --test-spec option)
> mbedgt: using '/Users/kvanb/git/ams-test-cyclic/.build/tests/LPC1768/GCC_ARM/test_spec.json' from current directory!
> mbedgt: detecting connected mbed-enabled devices...
> mbedgt: detected 1 device
> mbedgt: processing target 'LPC1768' toolchain 'GCC_ARM' compatible platforms... (note: switch set to --parallel 1)
> mbedgt: test case filter (specified with -n option)
>   test filtered in 'tests-sample-test-group-sample-test2'
> mbedgt: running 1 test for platform 'LPC1768' and toolchain 'GCC_ARM'
> mbedgt: mbed-host-test-runner: started
> mbedgt: checking for GCOV data...
> mbedgt: test case summary event not found
>   no test case report present, assuming test suite to be a single test case!
>   test suite: tests-sample-test-group-sample-test2
>   test case: group-sample-test2
> mbedgt: test on hardware with target id: 101000000000000000000002F7F214D673a6e4da75be38e5bd6bae3b02404c58
> mbedgt: test suite 'tests-sample-test-group-sample-test2' ............................................ TIMEOUT in 64.39 sec
>   test case: 'group-sample-test2' .............................................................. ERROR in 64.39 sec
> mbedgt: test case summary: 0 passes, 1 failure
> mbedgt: all tests finished!
> mbedgt: shuffle seed: 0.1821426898
> mbedgt: test suite report:
> +-----------------+---------------+--------------------------------------+---------+--------------------+-------------+
> | target          | platform_name | test suite                           | result  | elapsed_time (sec) | copy_method |
> +-----------------+---------------+--------------------------------------+---------+--------------------+-------------+
> | LPC1768-GCC_ARM | LPC1768       | tests-sample-test-group-sample-test2 | TIMEOUT | 64.39              | shell       |
> +-----------------+---------------+--------------------------------------+---------+--------------------+-------------+
> mbedgt: test suite results: 1 TIMEOUT
> mbedgt: test case report:
> +-----------------+---------------+--------------------------------------+--------------------+--------+--------+--------+---------> -----------+
> | target          | platform_name | test suite                           | test case          | passed | failed | result | > elapsed_time (sec) |
> +-----------------+---------------+--------------------------------------+--------------------+--------+--------+--------+---------> -----------+
> | LPC1768-GCC_ARM | LPC1768       | tests-sample-test-group-sample-test2 | group-sample-test2 | 0      | 1      | ERROR  | > 64.39              |
> +-----------------+---------------+--------------------------------------+--------------------+--------+--------+--------+---------> -----------+
> mbedgt: test case results: 1 ERROR
> mbedgt: completed in 64.58 sec
> mbedgt: exited with code 1
> [mbed] ERROR: "mbedgt" returned error code 1.
> [mbed] ERROR: Command "mbedgt --test-spec /Users/kvanb/git/ams-test-cyclic/.build/tests/LPC1768/GCC_ARM/test_spec.json -n > tests-sample-test-group-sample-test2" in "/Users/kvanb/git/ams-test-cyclic"

I think the mbed device ran out of space and that particular file name was working because the .bin file was already on the device, along with a LOT of other bin files. Any other file names were failing perhaps because the copy failed? Should this be a bug, perhaps it should report with a copy failed error message? Now that I have cleared out the lot of bin files it's working perfectly again with any file name

posted by Kyle Van Berendonck 24 Aug 2016

Could you file a bug at https://github.com/ARMmbed/mbed-os ?

posted by Jan Jongboom 24 Aug 2016

1 Answer

Willie Walker
7 years, 8 months ago.

Was a bug ever filed on this? I ran into this same problem today and spent many hours puzzling over it until I cam across this thread. Deleting all the *.bin's on the MBED device (LPC1768) resolved the problem.

No I never filed a bug for it sorry

posted by Kyle Van Berendonck 07 Apr 2017

I have just posted a bug for it https://github.com/ARMmbed/mbed-os/issues/4130

posted by Kyle Van Berendonck 07 Apr 2017

Hi everyone, I went ahead and replied on the GitHub issue. Let's continue the discussion over there.

posted by Brian Daniels 10 Apr 2017