Below, we share the Guideline-Enhanced prompt that we used in the article.
- Syntax:
- nsure YAML file follows proper syntax, including required symbols, e.g., colons, hyphens, or quotation marks.
- Verify that GitHub Actions YAML file follows proper indentation and structural conventions.
- Ensure all used definitions are correctly defined and properly placed.
- Ensure the value types match their expected format when such fields exist in the source configuration, e.g., the setup-python action expects a single version value, not a list.
- Platform Discrepancy:
- Translate Travis CI keys only when a clear, direct GitHub Actions equivalent exists. Skip any Travis CI–specific or unsupported keys.
- Verify that all expressions are properly converted to GitHub Actions syntax, e.g., 'env: TOX_ENV=flake8' to 'TOX_ENV: flake8'.
- Retain only x64 and arm64 if architectures are explicitly defined; remove all other architectures.
- Ensure all version numbers are in string format, e.g., Python '3.10'.
- Explicitly configure fields that are implicitly defaulted in Travis CI as equivalent GitHub Actions steps, e.g., actions/checkout@v4 and language runtime setup.
- Ensure all packages are properly installed and configured when present in the Travis CI configuration, e.g., migrate tox-travis to tox-gh-actions if applicable.
- Environment:
- Ensure all referenced GitHub Actions currently exist; avoid non-existent actions.
- Ensure all required tokens are properly configured for the migrated steps, e.g., set GITHUB_TOKEN in the coveralls step if such a step exists.
- Logic:
- Ensure all trigger events are correctly translated, e.g., avoid using tags as the sole trigger event unless the Travis CI configuration explicitly relies on tag-only builds.
- Ensure all execution orders are correctly translated, e.g., deploy job should be configured with the `needs` parameter to execute after test job.
- Ensure conditional statements are correctly translated.
- Do not introduce extra jobs, steps, operating systems, or matrix dimensions beyond those implied by the Travis CI configuration, except where required to faithfully reproduce Travis CI default behavior.
- Ensure that all transferable steps have been migrated.
| Metric | Importer | Basic | One-shot | Guideline | IR | Guideline+IR |
|---|---|---|---|---|---|---|
| BSR | 31.5% | 63.0% | 57.0% | 71.5% | 83.5% | 88.5% |
| LMR | 100.0% | 100.0% | 95.6% | 98.6% | 97.0% | 97.2% |
| Strategies | 0 | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|---|
| IR | 126 | 155 | 164 | 167 | 167 | 167 |
| Guideline+IR | 143 | 165 | 170 | 176 | 177 | 177 |
Our dataset comprises 200 GitHub repositories, and the source code is publicly available on citranslation.