Recently there was a regression in Pika, and it would generally be nice
to use distribution packages for most of our dependencies. Since the
image tester is already committing similar crimes I think it's fine for
the uploader, too.
Right now we only pull in the Google SDK bits from PyPI, as well as the
Azure compute module from there since the Azure CLI pins an ancient
version.
Signed-off-by: Jeremy Cline <jeremycline@microsoft.com>
Somewhat recently, Azure images started failing to upload, crashing when
calling azure_compute_client.gallery_images.begin_create_or_update with
a server-side error of "HttpResponseError: (InvalidParameter) Required
parameter 'galleryImageVersionName' is missing (null)."
It _seems_ like it no longer allows updating a gallery image (even if
there are no actual updates happening, it just declares it as it always
was) with features that don't include a version within that gallery
where the feature is first enabled.
Signed-off-by: Jeremy Cline <jeremycline@microsoft.com>
SDKs have been moving away from bundling all the API versions so this
isn't going to keep working.
Signed-off-by: Jeremy Cline <jeremycline@microsoft.com>
If we list the git repo installing a wheel of this package fails. We
have some silly work-arounds for LISA not being on PyPI (but really
really need to go chase that down).
Signed-off-by: Jeremy Cline <jeremycline@microsoft.com>
None of these tests actually work since I reworked things to deal with
the long test times and AMQP ack timeouts. They also reference a lot of
since-deleted code. It'd obviously be good to have working tests, but no
tests are better than confusingly wrong tests.
Signed-off-by: Jeremy Cline <jeremycline@microsoft.com>
Not totally sure about this one, I've removed a couple of test
conditions where it seems like an image was expected to be filtered out
but the only thing which has changed since the previous line was
removing the thing that caused it to be filtered in the first place.
Happy to put these back if someone can explain what the intention was.
GA and RC composes aren't formatted as a date. Pull the value from the
metadata instead, and fall back to the current time if that fails.
Signed-off-by: Jeremy Cline <jeremycline@microsoft.com>
This puts Rawhide and ELN under the Fedora directory in the storage
container; they already both include Rawhide/ELN in their path so it
should be okay to not replicate that.
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
Maintain composes.json along with junit and html results to the azure
blob sotrage. This enables to query test results programmatically for
the dashboard
Signed-off-by: Bala Konda Reddy M <bala12352@gmail.com>
This includes two fixes for Fedora test failures, one of which we were
carrying as a patch (to work with systemd 258+). The second fixes the
enable kprobe test.
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
Running the tests can take some time, and often runs up against the
maximum time RabbitMQ will wait for a message to be acknowledged. To
work around this, we have to defer running the tests to a thread and
immediately return from our callback so the library acknowledges the
message for us.
This means that if the consumer fails (or is restarted) we won't get a
redelivery of the message. However, images are built daily so losing one
day's test results isn't the end of the world. To ensure the loss of
messages isn't too great, the consumer sets up a queue of size one and
periodically nacks the next message. This is not exactly elegant, but to
"do it right" we'd need to use the AMQP library directly.
Currently many tests fail due to systemd introducing ANSI escape
sequences in logs the tests are inspecting. We should be able to drop
this patch whenever there's a new LISA release.
In the event the test run is interrupted (the pod is redeployed, for
example), it'll leave a bunch of expensive VMs running forever. This
adds a "delete" tag to resources LISA makes and does a check each run
for any stale resources.
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
The readme in the image tester package didn't specify that the config
file and containerfile are in the repository root. Also, add an example
of reconsuming a message for testing purposes.
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
This loads the credentials from a number of sources (azure-cli login,
environment variables, etc).
For production, we expect the AZURE_CLIENT_ID, AZURE_TENANT_ID, and
AZURE_CLIENT_SECRET or AZURE_CLIENT_CERTIFICATE_PATH variables to be
set.
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
Without this, it'll attempt to write in the default working directory
for the container, which is not write-able in environments like
OpenShift.
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
The parameter changed from "id" to "storage_account_id" when uploading,
presumably because the id field is used for the resource itself and not
the storage account. This breaking change was recently enforced server
side.
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
In order to make it easier to browse results, write out an index file to
the storage container.
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
Fixes up the Containerfile so it installs properly. This also ensures
the resource groups LISA creates are tagged so they can be easily
cleaned up asynchronously if LISA fails to clean up for whatever reason
(e.g. the container is restarted during a run).
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
Rather than distributing the two message schema separately, just keep
them in one package. The name is not quite as accurate now, but it's a
lot less work.
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
This commit introduces a new fedora-messaging consumer which runs LISA
tests against uploaded Azure images.
Signed-off-by: Bala Konda Reddy M <bala12352@gmail.com>
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
This is in preparation to add a garbage collection task for our Azure
subscription like the AWS account has. Resources including a "Delete"
tag with a date will be deleted after that date.
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
