Below, we share the Guideline-Enhanced prompt that we used in the article.
- Syntax:
- Ensure YAML file follows proper syntax, including required symbols, e.g., colons, hyphens, or quotation marks.
- Verify that GitHub Actions YAML file follows proper indentation and structural conventions.
- Ensure all used definitions are correctly defined and properly placed.
- Ensure the value types match their expected format, e.g., the setup-python action expects a single version value, not a list.
- Platform Discrepancy:
- Ensure all Travis CI keys are translated to GitHub Actions keys. Travis CI specific keys , e.g., notification and allow_failure, may be discarded.
- Verify that all expressions are properly converted to GitHub Actions syntax, e.g., 'env: TOX_ENV=flake8' to 'TOX_ENV: flake8'.
- Retain only x64 and arm64; remove all other architectures.
- Ensure all version numbers are in string format, e.g., Python '3.10'.
- Explicitly configure fields that are implicitly defaulted in Travis CI, e.g., actions/checkout@v2, actions/setup-python@v2, and test environment setup.
- Ensure all referenced dependencies are present.
- Environment:
- Ensure all actions are currently supported, e.g., actions/cache v3 or higher.
- Ensure all required tokens are properly configured, e.g., set GITHUB_TOKEN in coveralls step.
- Logic:
- Ensure all trigger events are correctly translated, e.g., avoid using tags as the sole trigger event.
- Ensure all execution orders are correctly translated, e.g., deploy job should be configured with the `needs` parameter to execute after test job.
- Ensure conditional statements are correctly translated.
- Ensure that all transferable steps have been migrated.
- Ensure all tasks are translated without introducing additional or unrelated tasks.
| Metric | Importer | Basic | One-shot | Guideline | IR | Guideline+IR |
|---|---|---|---|---|---|---|
| BSR | 18.3% | 25.8% | 21.0% | 40.2% | 68.6% | 75.5% |
| CS | 57.7% | 72.4% | 70.7% | 73.3% | 72.5% | 71.7% |
| CB | 6.1% | 29.0% | 26.2% | 28.2% | 27.9% | 27.1% |
Our dataset comprises 229 GitHub repositories, and the source code is publicly available on citranslation.