the other jimmy / mbed-sdk-tools

Important changes to repositories hosted on mbed.com

Mbed hosted mercurial repositories are deprecated and are due to be permanently deleted in July 2026.

To keep a copy of this software download the repository Zip archive or clone locally using Mercurial.

It is also possible to export all your personal repositories from the account settings page.

Fork of mbed-sdk-tools by mbed official

test/examples/update.py@31:182518299918, 2017-01-04 (annotated)

Committer:
The Other Jimmy
Date:
Wed Jan 04 11:58:24 2017 -0600
Revision:
31:182518299918
Update tools to follow mbed-os tools release 5.3.1

Who changed what in which revision?

UserRevisionLine numberNew contents of line
The Other Jimmy 31:182518299918 1 #!/usr/bin/env python
The Other Jimmy 31:182518299918 2
The Other Jimmy 31:182518299918 3 import os
The Other Jimmy 31:182518299918 4 from os.path import dirname, abspath, basename
The Other Jimmy 31:182518299918 5 import sys
The Other Jimmy 31:182518299918 6 import argparse
The Other Jimmy 31:182518299918 7 import json
The Other Jimmy 31:182518299918 8 import subprocess
The Other Jimmy 31:182518299918 9 import shutil
The Other Jimmy 31:182518299918 10 import stat
The Other Jimmy 31:182518299918 11
The Other Jimmy 31:182518299918 12 ROOT = abspath(dirname(dirname(dirname(dirname(__file__)))))
The Other Jimmy 31:182518299918 13 sys.path.insert(0, ROOT)
The Other Jimmy 31:182518299918 14
The Other Jimmy 31:182518299918 15 import examples_lib as lib
The Other Jimmy 31:182518299918 16 from examples_lib import SUPPORTED_TOOLCHAINS
The Other Jimmy 31:182518299918 17
The Other Jimmy 31:182518299918 18 def run_cmd(command, print_warning_on_fail=True):
The Other Jimmy 31:182518299918 19 """ Takes the command specified and runs it in a sub-process, obtaining the return code.
The Other Jimmy 31:182518299918 20
The Other Jimmy 31:182518299918 21 Args:
The Other Jimmy 31:182518299918 22 command - command to run, provided as a list of individual fields which are combined into a
The Other Jimmy 31:182518299918 23 single command before passing to the sub-process call.
The Other Jimmy 31:182518299918 24 return_code - result of the command.
The Other Jimmy 31:182518299918 25
The Other Jimmy 31:182518299918 26 """
The Other Jimmy 31:182518299918 27 print('[Exec] %s' % ' '.join(command))
The Other Jimmy 31:182518299918 28 return_code = subprocess.call(command)
The Other Jimmy 31:182518299918 29
The Other Jimmy 31:182518299918 30 if return_code:
The Other Jimmy 31:182518299918 31 print("The command '%s' failed with return code: %s" % (' '.join(command), return_code))
The Other Jimmy 31:182518299918 32 print("Ignoring and moving on to the next example")
The Other Jimmy 31:182518299918 33
The Other Jimmy 31:182518299918 34 return return_code
The Other Jimmy 31:182518299918 35
The Other Jimmy 31:182518299918 36
The Other Jimmy 31:182518299918 37 def rmtree_readonly(directory):
The Other Jimmy 31:182518299918 38 """ Deletes a readonly directory tree.
The Other Jimmy 31:182518299918 39
The Other Jimmy 31:182518299918 40 Args:
The Other Jimmy 31:182518299918 41 directory - tree to delete
The Other Jimmy 31:182518299918 42 """
The Other Jimmy 31:182518299918 43 def remove_readonly(func, path, _):
The Other Jimmy 31:182518299918 44 os.chmod(path, stat.S_IWRITE)
The Other Jimmy 31:182518299918 45 func(path)
The Other Jimmy 31:182518299918 46
The Other Jimmy 31:182518299918 47 shutil.rmtree(directory, onerror=remove_readonly)
The Other Jimmy 31:182518299918 48
The Other Jimmy 31:182518299918 49 def find_all_examples(path):
The Other Jimmy 31:182518299918 50 """ Searches the path specified for sub-example folders, ie those containing an
The Other Jimmy 31:182518299918 51 mbed-os.lib file. If found adds the path to the sub-example to a list which is
The Other Jimmy 31:182518299918 52 then returned.
The Other Jimmy 31:182518299918 53
The Other Jimmy 31:182518299918 54 Args:
The Other Jimmy 31:182518299918 55 path - path to search.
The Other Jimmy 31:182518299918 56 examples - (returned) list of paths to example directories.
The Other Jimmy 31:182518299918 57
The Other Jimmy 31:182518299918 58 """
The Other Jimmy 31:182518299918 59 examples = []
The Other Jimmy 31:182518299918 60 for root, dirs, files in os.walk(path):
The Other Jimmy 31:182518299918 61 if 'mbed-os.lib' in files:
The Other Jimmy 31:182518299918 62 examples += [root]
The Other Jimmy 31:182518299918 63
The Other Jimmy 31:182518299918 64 return examples
The Other Jimmy 31:182518299918 65
The Other Jimmy 31:182518299918 66 def upgrade_single_example(example, tag, directory):
The Other Jimmy 31:182518299918 67 """ Updates the mbed-os.lib file in the example specified to correspond to the
The Other Jimmy 31:182518299918 68 version specified by the GitHub tag supplied. Also deals with
The Other Jimmy 31:182518299918 69 multiple sub-examples in the GitHub repo, updating them in the same way.
The Other Jimmy 31:182518299918 70
The Other Jimmy 31:182518299918 71 Args:
The Other Jimmy 31:182518299918 72 example - json example object containing the GitHub repo to update.
The Other Jimmy 31:182518299918 73 tag - GitHub tag corresponding to a version of mbed-os to upgrade to.
The Other Jimmy 31:182518299918 74 directory - directory path for the example.
The Other Jimmy 31:182518299918 75 returns - True if the upgrade was successful, False otherwise.
The Other Jimmy 31:182518299918 76
The Other Jimmy 31:182518299918 77 """
The Other Jimmy 31:182518299918 78 print("Upgrading single example at path '%s'" % directory)
The Other Jimmy 31:182518299918 79 cwd = os.getcwd()
The Other Jimmy 31:182518299918 80 os.chdir(directory)
The Other Jimmy 31:182518299918 81
The Other Jimmy 31:182518299918 82 return_code = None
The Other Jimmy 31:182518299918 83
The Other Jimmy 31:182518299918 84 # Change directories to the mbed-os library
The Other Jimmy 31:182518299918 85 if not os.path.exists('mbed-os'):
The Other Jimmy 31:182518299918 86 print("'mbed-os' directory not found in the root of '%s'" % directory)
The Other Jimmy 31:182518299918 87 print("Ignoring and moving on to the next example")
The Other Jimmy 31:182518299918 88 os.chdir(cwd)
The Other Jimmy 31:182518299918 89 return False
The Other Jimmy 31:182518299918 90
The Other Jimmy 31:182518299918 91 os.chdir('mbed-os')
The Other Jimmy 31:182518299918 92
The Other Jimmy 31:182518299918 93 # Setup and run the update command
The Other Jimmy 31:182518299918 94 update_cmd = ['mbed', 'update', tag]
The Other Jimmy 31:182518299918 95 return_code = run_cmd(update_cmd)
The Other Jimmy 31:182518299918 96
The Other Jimmy 31:182518299918 97 if return_code:
The Other Jimmy 31:182518299918 98 os.chdir(cwd)
The Other Jimmy 31:182518299918 99 return False
The Other Jimmy 31:182518299918 100
The Other Jimmy 31:182518299918 101 os.chdir('../')
The Other Jimmy 31:182518299918 102
The Other Jimmy 31:182518299918 103 # Setup and run the add command
The Other Jimmy 31:182518299918 104 add_cmd = ['git', 'add', 'mbed-os.lib']
The Other Jimmy 31:182518299918 105 return_code = run_cmd(add_cmd)
The Other Jimmy 31:182518299918 106
The Other Jimmy 31:182518299918 107 os.chdir(cwd)
The Other Jimmy 31:182518299918 108 return not return_code
The Other Jimmy 31:182518299918 109
The Other Jimmy 31:182518299918 110 def upgrade_example(example, tag):
The Other Jimmy 31:182518299918 111 """ Clones the example specified from GitHub and updates the associated mbed-os.lib file
The Other Jimmy 31:182518299918 112 to correspond to the version specified by the GitHub tag supplied. Also deals with
The Other Jimmy 31:182518299918 113 multiple sub-examples in the GitHub repo, updating them in the same way.
The Other Jimmy 31:182518299918 114
The Other Jimmy 31:182518299918 115 Args:
The Other Jimmy 31:182518299918 116 example - json example object containing the GitHub repo to update.
The Other Jimmy 31:182518299918 117 tag - GitHub tag corresponding to a version of mbed-os to upgrade to.
The Other Jimmy 31:182518299918 118
The Other Jimmy 31:182518299918 119 """
The Other Jimmy 31:182518299918 120 print("Updating example '%s'" % example['name'])
The Other Jimmy 31:182518299918 121 cwd = os.getcwd()
The Other Jimmy 31:182518299918 122
The Other Jimmy 31:182518299918 123 # Setup and run the import command
The Other Jimmy 31:182518299918 124 clone_cmd = ['git', 'clone', example['github']]
The Other Jimmy 31:182518299918 125 return_code = run_cmd(clone_cmd)
The Other Jimmy 31:182518299918 126
The Other Jimmy 31:182518299918 127 if return_code:
The Other Jimmy 31:182518299918 128 return False
The Other Jimmy 31:182518299918 129
The Other Jimmy 31:182518299918 130 # Find all examples
The Other Jimmy 31:182518299918 131 example_directories = find_all_examples(example['name'])
The Other Jimmy 31:182518299918 132
The Other Jimmy 31:182518299918 133 os.chdir(example['name'])
The Other Jimmy 31:182518299918 134
The Other Jimmy 31:182518299918 135 # Setup and run the update command
The Other Jimmy 31:182518299918 136 import_cmd = ['mbed', 'update']
The Other Jimmy 31:182518299918 137 return_code = run_cmd(import_cmd)
The Other Jimmy 31:182518299918 138 if return_code:
The Other Jimmy 31:182518299918 139 os.chdir(cwd)
The Other Jimmy 31:182518299918 140 return False
The Other Jimmy 31:182518299918 141
The Other Jimmy 31:182518299918 142 for example_directory in example_directories:
The Other Jimmy 31:182518299918 143 if not upgrade_single_example(example, tag, os.path.relpath(example_directory, example['name'])):
The Other Jimmy 31:182518299918 144 os.chdir(cwd)
The Other Jimmy 31:182518299918 145 return False
The Other Jimmy 31:182518299918 146
The Other Jimmy 31:182518299918 147 # Setup the default commit message
The Other Jimmy 31:182518299918 148 commit_message = 'Updating mbed-os to {{' + tag +'}}'
The Other Jimmy 31:182518299918 149
The Other Jimmy 31:182518299918 150 # Setup and run the commit command
The Other Jimmy 31:182518299918 151 commit_cmd = ['git', 'commit', '-m', commit_message]
The Other Jimmy 31:182518299918 152 return_code = run_cmd(commit_cmd)
The Other Jimmy 31:182518299918 153 if return_code:
The Other Jimmy 31:182518299918 154 if return_code == 1:
The Other Jimmy 31:182518299918 155 print("[WARNING] 'git commit' exited with a return code of 1. " + \
The Other Jimmy 31:182518299918 156 "This usually inidicates that no update was made. Still " + \
The Other Jimmy 31:182518299918 157 "attempting to create a tag.")
The Other Jimmy 31:182518299918 158 else:
The Other Jimmy 31:182518299918 159 os.chdir(cwd)
The Other Jimmy 31:182518299918 160 return False
The Other Jimmy 31:182518299918 161
The Other Jimmy 31:182518299918 162 # Setup and run the tag command
The Other Jimmy 31:182518299918 163 tag_cmd = ['git', 'tag', '-a', tag, '-m', tag]
The Other Jimmy 31:182518299918 164 return_code = run_cmd(tag_cmd)
The Other Jimmy 31:182518299918 165 if return_code:
The Other Jimmy 31:182518299918 166 os.chdir(cwd)
The Other Jimmy 31:182518299918 167 return False
The Other Jimmy 31:182518299918 168
The Other Jimmy 31:182518299918 169 # Setup and run the push command
The Other Jimmy 31:182518299918 170 push_cmd = ['git', 'push', 'origin', 'master']
The Other Jimmy 31:182518299918 171 return_code = run_cmd(push_cmd)
The Other Jimmy 31:182518299918 172
The Other Jimmy 31:182518299918 173 if return_code:
The Other Jimmy 31:182518299918 174 os.chdir(cwd)
The Other Jimmy 31:182518299918 175 return False
The Other Jimmy 31:182518299918 176
The Other Jimmy 31:182518299918 177 push_cmd = ['git', 'push', 'origin', tag]
The Other Jimmy 31:182518299918 178 return_code = run_cmd(push_cmd)
The Other Jimmy 31:182518299918 179
The Other Jimmy 31:182518299918 180 os.chdir(cwd)
The Other Jimmy 31:182518299918 181 return not return_code
The Other Jimmy 31:182518299918 182
The Other Jimmy 31:182518299918 183 def create_work_directory(path):
The Other Jimmy 31:182518299918 184 """ Create a new directory specified in 'path', overwrite if the directory already
The Other Jimmy 31:182518299918 185 exists.
The Other Jimmy 31:182518299918 186
The Other Jimmy 31:182518299918 187 Args:
The Other Jimmy 31:182518299918 188 path - directory path to be created.
The Other Jimmy 31:182518299918 189
The Other Jimmy 31:182518299918 190 """
The Other Jimmy 31:182518299918 191 if os.path.exists(path):
The Other Jimmy 31:182518299918 192 print("'%s' directory already exists. Deleting..." % path)
The Other Jimmy 31:182518299918 193 rmtree_readonly(path)
The Other Jimmy 31:182518299918 194
The Other Jimmy 31:182518299918 195 os.makedirs(path)
The Other Jimmy 31:182518299918 196
The Other Jimmy 31:182518299918 197 def test_compile(config, tag):
The Other Jimmy 31:182518299918 198 """ For each example repo identified in the config json object, clone, update mbed-os to
The Other Jimmy 31:182518299918 199 the specified tag and then compile for all supported toolchains.
The Other Jimmy 31:182518299918 200
The Other Jimmy 31:182518299918 201 Args:
The Other Jimmy 31:182518299918 202 config - the json object imported from the file.
The Other Jimmy 31:182518299918 203 tag - GitHub tag corresponding to a version of mbed-os to upgrade to.
The Other Jimmy 31:182518299918 204 results - summary of compilation results.
The Other Jimmy 31:182518299918 205
The Other Jimmy 31:182518299918 206 """
The Other Jimmy 31:182518299918 207 # Create work directories
The Other Jimmy 31:182518299918 208 create_work_directory('test_compile')
The Other Jimmy 31:182518299918 209
The Other Jimmy 31:182518299918 210 # Loop through the examples
The Other Jimmy 31:182518299918 211 results = {}
The Other Jimmy 31:182518299918 212 os.chdir('test_compile')
The Other Jimmy 31:182518299918 213
The Other Jimmy 31:182518299918 214 lib.source_repos(config)
The Other Jimmy 31:182518299918 215 lib.update_mbedos_version(config, tag)
The Other Jimmy 31:182518299918 216 results = lib.compile_repos(config, SUPPORTED_TOOLCHAINS)
The Other Jimmy 31:182518299918 217 os.chdir("..")
The Other Jimmy 31:182518299918 218
The Other Jimmy 31:182518299918 219 return results
The Other Jimmy 31:182518299918 220
The Other Jimmy 31:182518299918 221
The Other Jimmy 31:182518299918 222 def main(arguments):
The Other Jimmy 31:182518299918 223
The Other Jimmy 31:182518299918 224 parser = argparse.ArgumentParser(description=__doc__,
The Other Jimmy 31:182518299918 225 formatter_class=argparse.RawDescriptionHelpFormatter)
The Other Jimmy 31:182518299918 226 parser.add_argument('tag', help="mbed-os tag to which all examples will be updated")
The Other Jimmy 31:182518299918 227 parser.add_argument('-c', '--config_file', help="Path to the configuration file (default is 'examples.json')", default='examples.json')
The Other Jimmy 31:182518299918 228
The Other Jimmy 31:182518299918 229 args = parser.parse_args(arguments)
The Other Jimmy 31:182518299918 230
The Other Jimmy 31:182518299918 231 cfg = os.path.join(os.path.dirname(__file__), args.config_file)
The Other Jimmy 31:182518299918 232
The Other Jimmy 31:182518299918 233 # Load the config file
The Other Jimmy 31:182518299918 234 config = json.load(open(os.path.join(os.path.dirname(__file__),
The Other Jimmy 31:182518299918 235 args.config_file)))
The Other Jimmy 31:182518299918 236
The Other Jimmy 31:182518299918 237 if not config:
The Other Jimmy 31:182518299918 238 print("Failed to load config file '%s'" % args.config_file)
The Other Jimmy 31:182518299918 239 sys.exit(1)
The Other Jimmy 31:182518299918 240
The Other Jimmy 31:182518299918 241 # Create work directories
The Other Jimmy 31:182518299918 242 create_work_directory('examples')
The Other Jimmy 31:182518299918 243
The Other Jimmy 31:182518299918 244 # Loop through the examples
The Other Jimmy 31:182518299918 245 failures = []
The Other Jimmy 31:182518299918 246 successes = []
The Other Jimmy 31:182518299918 247 not_compiled = []
The Other Jimmy 31:182518299918 248 results = {}
The Other Jimmy 31:182518299918 249 os.chdir('examples')
The Other Jimmy 31:182518299918 250
The Other Jimmy 31:182518299918 251 results = test_compile(config, args.tag)
The Other Jimmy 31:182518299918 252 lib.print_compilation_summary(results)
The Other Jimmy 31:182518299918 253
The Other Jimmy 31:182518299918 254 for example in config['examples']:
The Other Jimmy 31:182518299918 255 # Determine if this example should be updated
The Other Jimmy 31:182518299918 256
The Other Jimmy 31:182518299918 257 # Attempt to update if:
The Other Jimmy 31:182518299918 258 # group of examples passed compilation and
The Other Jimmy 31:182518299918 259 # auto update is set to True
The Other Jimmy 31:182518299918 260 # Note: results fields are [compiled flag, pass flag, successes list, failures list]
The Other Jimmy 31:182518299918 261 if not results[example['name']][0]:
The Other Jimmy 31:182518299918 262 # Example was not compiled
The Other Jimmy 31:182518299918 263 not_compiled += [example['name']]
The Other Jimmy 31:182518299918 264 else:
The Other Jimmy 31:182518299918 265 if results[example['name']][1] and example['auto-update']:
The Other Jimmy 31:182518299918 266 if upgrade_example(example, args.tag):
The Other Jimmy 31:182518299918 267 successes += [example['name']]
The Other Jimmy 31:182518299918 268 else:
The Other Jimmy 31:182518299918 269 failures += [example['name']]
The Other Jimmy 31:182518299918 270 else:
The Other Jimmy 31:182518299918 271 failures += [example['name']]
The Other Jimmy 31:182518299918 272
The Other Jimmy 31:182518299918 273 os.chdir('../')
The Other Jimmy 31:182518299918 274
The Other Jimmy 31:182518299918 275 # Finish the script and report the results
The Other Jimmy 31:182518299918 276 print(os.linesep + os.linesep +'Finished updating examples!' + os.linesep)
The Other Jimmy 31:182518299918 277
The Other Jimmy 31:182518299918 278 if successes:
The Other Jimmy 31:182518299918 279 print('The following examples updated successfully:')
The Other Jimmy 31:182518299918 280 for success in successes:
The Other Jimmy 31:182518299918 281 print(' - %s' % success)
The Other Jimmy 31:182518299918 282
The Other Jimmy 31:182518299918 283 if failures:
The Other Jimmy 31:182518299918 284 print('\nThe following examples were not updated:')
The Other Jimmy 31:182518299918 285 for fail in failures:
The Other Jimmy 31:182518299918 286 print(' - %s' % fail)
The Other Jimmy 31:182518299918 287
The Other Jimmy 31:182518299918 288 if not_compiled:
The Other Jimmy 31:182518299918 289 print('The following examples were skipped:')
The Other Jimmy 31:182518299918 290 for example in not_compiled:
The Other Jimmy 31:182518299918 291 print(' - %s' % example)
The Other Jimmy 31:182518299918 292
The Other Jimmy 31:182518299918 293
The Other Jimmy 31:182518299918 294 if __name__ == '__main__':
The Other Jimmy 31:182518299918 295 sys.exit(main(sys.argv[1:]))