Commit Graph

14 Commits

Author SHA1 Message Date
Nicolas Lacasse d5b3dd7cb4 Run all runtime tests in a single container.
This makes them run much faster. Also cleaned up the log reporting.

PiperOrigin-RevId: 270799808
2019-09-23 17:43:42 -07:00
Nicolas Lacasse 28f431335b Shard the runtime tests.
Default of 20 shards was arbitrary and will need fine-tuning in later CLs.

PiperOrigin-RevId: 269922871
2019-09-18 17:04:53 -07:00
Nicolas Lacasse 062190d983 Follow-up fixes for image tests.
- Fix ARG syntax in Dockerfiles.
- Fix curl commands in Dockerfiles.
- Fix some paths in proctor binaries.
- Check error from Walk in search helper.

PiperOrigin-RevId: 269641686
2019-09-17 13:29:19 -07:00
Nicolas Lacasse 24b7eb2f86 Refactor and clean up image tests.
* Use multi-stage builds in Dockerfiles.
* Combine all proctor binaries into a single binary.
* Change the TestRunner interface to reduce code duplication.

PiperOrigin-RevId: 269462101
2019-09-16 17:51:22 -07:00
Adin Scannell 67a2ab1438 Impose order on test scripts.
The simple test script has gotten out of control. Shard this script into
different pieces and attempt to impose order on overall test structure. This
change helps lay some of the foundations for future improvements.

 * The runsc/test directories are moved into just test/.
 * The runsc/test/testutil package is split into logical pieces.
 * The scripts/ directory contains new top-level targets.
 * Each test is now responsible for building targets it requires.
 * The install functionality is moved into `runsc` itself for simplicity.
 * The existing kokoro run_tests.sh file now just calls all (can be split).

After this change is merged,  I will create multiple distinct workflows for
Kokoro, one for each of the scripts currently targeted by `run_tests.sh` today,
which should dramatically reduce the time-to-run for the Kokoro tests, and
provides a better foundation for further improvements to the infrastructure.

PiperOrigin-RevId: 267081397
2019-09-03 22:02:43 -07:00
Nicolas Lacasse eb4aa40342 Compile procter binaries during image creation.
Using "go run ..." in the ENTRYPOINT causes the go compiler to run each time
the container is started. We can just compile the binary once as part of the
image.

PiperOrigin-RevId: 266212462
2019-08-29 14:02:32 -07:00
Nicolas Lacasse 8b7e7a04d6 Don't run runtime tests in parallel.
We need real sharding, and will let Bazel handle the
parallelization. That is coming soon. Until then, remove
this call to t.Parallel() so that we can run the tests without
eating all CPU.

PiperOrigin-RevId: 264498919
2019-08-20 16:59:09 -07:00
Brett Landau f2762e8c60 Create tests for common.Search().
Using the path_test.go file built by the Golang
devs as a base, tests have been created to verify
the functionality of common.Search().

A mock file system is created and fake test files
are generated to see if they get picked up by
common.Search().

Also included in this CL is a bug fix for
proctor-nodejs that was discovered using this test.
proctor-nodejs used to allow multiple "-" in its
test name filter. The regex has been updated to
prevent this.

PiperOrigin-RevId: 262647263
2019-08-09 15:35:58 -07:00
Samantha Sample fb996668e4 Alter Dockerfiles to include common.go and use a prebuilt JDK.
After the refactoring of the proctor binaries, the Dockerfiles for each
language must be altered to copy the common folder into their image.
Additionally, Java has been changed to use the pre-built version of
JDK-11 from Ubuntu, instead of building it from the source.  This allows
for a smaller image and faster test execution within the container.

PiperOrigin-RevId: 261805158
2019-08-05 17:20:05 -07:00
Samantha Sample 23e740433d Expand runtimes test suite to include Go, Java, PHP, and Python.
This change adds functionality for running more languages using
the runtimes test suite. It divides the languages into separate
test functions, which each call the helper testLang function in the
runtimes_test.go file. This allows them to be run individually
or as a group.

PiperOrigin-RevId: 261791935
2019-08-05 16:11:18 -07:00
Brett Landau 6a1ac34077 Refactor ListTests() to common.Search().
This change removes the filepath.Walk() function from
proctor- go, php, and nodejs. The filepath.Walk() is
now defined in common.go in Search(). Each proctor binary
passes root directory and testFilter arguments to Search().

proctor-python.go no longer uses filepath.Walk() to search
for tests. There is a built-in list test function within
python's language test suite so that is being used instead.

PiperOrigin-RevId: 261242897
2019-08-01 18:53:35 -07:00
Brett Landau 77833ece3b Fix test execution bugs in proctor-go and proctor-python.
proctor-go had a bug where it would incorrectly identify
a tool test as a disk test. Instead of searching for the
test on disk as the identification method, we now check if
the test name ends in ".go". If the test ends in ".go" it
is run as a disk test, otherwise the test is run as a tool test.

Python tests need to be run from within the directory they exist.
Functionality to split the test name from it's parent directory
has been added and a cmd.Dir argument has been set.

PiperOrigin-RevId: 261021693
2019-07-31 16:30:07 -07:00
Brett Landau edcc60b931 Refactor proctor binaries to implement testRunner interface.
Shared code among proctor-*.go files has been refactored
into common/common.go. The common package is imported in
each proctor binary and a struct is created to implement
the testRunner interface defined in common.go. This allows
for the proctor binaries to be updated without having to
copy/paste the same code across all files. There are no
usage or functionality changes.

PiperOrigin-RevId: 260967080
2019-07-31 11:35:47 -07:00
Samantha Sample 8e8b609611 Move runtimes tests to appropriate directory.
PiperOrigin-RevId: 260577765
2019-07-29 14:17:11 -07:00