You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
confused with the latest changes - one branch? #46
It seems that this repo moved again from the branch system (one branch = one version) to all-in-one branch. It's a bit unclear to me, how I can fetch one version for packaging. And it's also unclear, how you'll tag the releases.
@perexg sorry again, we've had a lot of discussion on how to improve this with the SOF developers, and @marc-hb has been improving how we move forward here. We have a pending Readme patch to be merged before we tag today, i.e. the binaries are all correct but we will tag once the readme update is merged today.
Ok, but how I can fetch only v1.8 with the current scheme? If you tag the main branch, I'll fetch both 1.7 and 1.8. And if there are 10 releases, I'll end to fetch all of them.
The solution may be to filter the files in the release assets, but they have wrong names (v1.7.tar.bz instead sof-bin-1.7.tar.gz) or return back to the release branches.
Ok, but how I can fetch only v1.8 with the current scheme? If you tag the main branch, I'll fetch both 1.7 and 1.8. And if there are 10 releases, I'll end to fetch all of them.
The solution may be to filter the files in the release assets, but they have wrong names (v1.7.tar.bz instead sof-bin-1.7.tar.gz).
I think @marc-hb has some flows for installation. Lets wait until he's online later today.
@marc-hb thanks for your inputs here, are there any docs in sof-docs or GH feature request that describe the release process roadmap and your current plans. I assume the end goal is to have a script that can create a tar.bz2 of FW, ldcs and topolgies for someone to attach to GH releases ?
You can't do that anymore. The longer term fix for that is to move away from using git to release binaries. For now the entire .git/ folder is 10M.
Just a note: From the storage perspective, it does not matter, if you store different source binary files to one branch or multiple branches. Perhaps, git merging may help, too.
I'm using the auto-generated tar balls by github now (using the commit-id not your tag which was floating previously). In this case, only the necessary files are downloaded. Your move to artifacts means that you plan to use own scripts for the release, which is also fine, but the storage can be used efficiently with the branches, too like:
main branch (without binary files, only license files and scripts)
version branch (with binary files for the specific version + merged main branch)
In this case, you can just tag the version branch and you'll get the automatic tar ball from github.
Or you may create meta branch with scripts and doc files which will be merged to main (all branches) and to separate release branches. There are multiple possibilities.
It's still missing a few features: final tarball, checksums, compilation flags,... I was hoping to get some user feedback to help prioritize further development but I'm afraid most people still use xtensa-build-all.sh + build-tools.sh + their own, private scripts on top. Cause why change? This is also the case of our internal Jenkins-based build and validation; this mismatch makes each release time-consuming and error-prone.
It's been unsurprisingly hard to get people excited about packaging and releases so the only way forward has been slow: try something (not too) different and wait a bit to see if anyone is confused or complains. There :-)
... for someone to attach to GH releases ?
Yes, my idea was to eventually just upload the output of the installer directly to https://docs.github.com/en/github/administering-a-repository/releasing-projects-on-github/managing-releases-in-a-repository and deprecate the entire sof-bin repo: everything required to perform releases would be self-contained in the main sof repo, no external dependencies and no other branches or tag besides the "real" sof ones. Hopefully simple enough not to require any Powerpoint: configure and run the installer/, upload to github releases, done. No secret sauce hidden in Jenkins or wherever.
I'm using the auto-generated tar balls by github now (using the commit-id not your tag which was floating previously). In this case, only the necessary files are downloaded.
First, thanks for this clarification and your expert feedback in general: even more useful considering how rare you are.
I didn't even know github had such a convenient git archive implementation, sorry I "broke" that. I understand the convenience of tagging sof-bin now, I will do today.
Once I tag sof-bin this git archive feature should not really be "broken" any more, just less efficient. Do you mind ignoring the older versions until we move to github releases? Just a few spurious megabytes every few months.
For the record I "announced" the move to a single branch in January 2021 (among many other things, some abandoned). I understand that didn't make any headline :-) thesofproject/sof#3775 (comment)
FWIW we added recently a pointer to sof-bin in the drivers:
sound/soc/sof/loader.c: "you may need to download the firmware from https://github.com/thesofproject/sof-bin/\n");
sound/soc/sof/topology.c: "you may need to download the firmware from https://github.com/thesofproject/sof-bin/\n");
you may change directions on how you want to deliver the binaries but that site has to stay.
I think that you can upload the manually generated assets to the release tag until you have the final solution.
Good idea and close enough to the final solution anyway; just in a different repo for now. Will do after README.md #37 is merged and after I create the v1.8-rc2 tag in sof-bin.
From my packager perspective, I would like just to change the source package URL in my rpm spec file
Thanks, I think I understand better now. In fact I just found https://koji.fedoraproject.org/koji/packageinfo?packageID=30869. I ran dnf download --source alsa-sof-firmware on some Fedora box and had a look at the .spec and other files. I should have done this earlier.
Do not forget that we store a copy of the source packages in our build system forever, too.
Are you expressing a space concerning here and/or other(s)?
Just to be extra sure: in this particular case, "source package" means "sof-bin sources" which means firmware binaries, correct?
I think that you can upload the manually generated assets to the release tag until you have the final solution.
Good idea and close enough to the final solution anyway; just in a different repo for now. Will do after README.md #37 is merged and after I create the v1.8-rc2 tag in sof-bin.
From my packager perspective, I would like just to change the source package URL in my rpm spec file
Thanks, I think I understand better now. In fact I just found https://koji.fedoraproject.org/koji/packageinfo?packageID=30869. I ran dnf download --source alsa-sof-firmware on some Fedora box and had a look at the .spec and other files. I should have done this earlier.
Here is a straight change from the commit-id to the version style for 1.7 (sitting in my private repo until we know the final source URL for 1.8):
--- a/alsa-sof-firmware.spec
+++ b/alsa-sof-firmware.spec
@@ -4,8 +4,6 @@
%global _firmwarepath /usr/lib/firmware
%global sof_version 1.7
-%global sof_commit 6bd5501e6ac3f29dde42a28ffe03a2942aa60640
-%global sof_shortcommit %(c=%{sof_commit}; echo ${c:0:7})
%global tplg_version 1.2.4
@@ -16,7 +14,7 @@ Release: 1%{?dist}
# See later in the spec for a breakdown of licensing
License: BSD
URL: https://github.com/thesofproject/sof-bin
-Source: https://github.com/thesofproject/sof-bin/archive/%{sof_commit}/sof-bin-%{sof_shortcommit}.tar.gz
+Source: https://github.com/thesofproject/sof-bin/archive/v%{sof_version}/sof-bin-%{sof_version}.tar.gz
BuildRequires: alsa-topology >= %{tplg_version}
BuildRequires: alsa-topology-utils >= %{tplg_version}
Conflicts: alsa-firmware <= 1.2.1-6
@@ -36,7 +34,7 @@ License: BSD
This package contains the debug files for the Sound Open Firmware project.
%prep
-%autosetup -n sof-bin-%{sof_commit}
+%autosetup -n sof-bin-%{sof_version}
mkdir -p firmware/intel/sof
Do not forget that we store a copy of the source packages in our build system forever, too.
Are you expressing a space concerning here and/or other(s)?
I only mean that the source package contents propagate to all distributions which need to deal with it somehow. So removing duplicates is really welcome.
Just to be extra sure: in this particular case, "source package" means "sof-bin sources" which means firmware binaries, correct?
Yes, I mean the source for the packaging here (thus the sof-bin tar or zip archive) - Source: line in the rpm spec file.
Thank you for your attention and your work on this.
@mwasko who from your team can work with @marc-hb on aligning the signing process with the release process to make it less manual and error prone. I would imagine that some helper scripts could be used to take the signed output from the signing system and upload/copy to the release destination. This can be done for v1.9 so we have 3 months.
So this should look exactly like v1.7 and should let you get back to a single %{sof_version} variable, please confirm.
I just updated the sof-bin-v1.8-rc2.tar.gz accordingly. Note: I normally never change anything released and I'm surprised github let me do that without even a warning. However this is a release candidate and it's a new approach that I uploaded less than one hour ago. So I'm pretty confident you've been the only user so far :-)
@mwasko who from your team can work with @marc-hb on aligning the signing process with the release process to make it less manual and error prone. I would imagine that some helper scripts could be used to take the signed output from the signing system and upload/copy to the release destination. This can be done for v1.9 so we have 3 months.
@slawblauciak and me are coordinating sof releases. I would recommend to prepare/extend script to validate release content and report if anything is missing. We can also work on some helper scripts to upload signed binaries, but there will be still some manual work dependency.
Activity
lgirdwood commentedon Jun 3, 2021
@perexg sorry again, we've had a lot of discussion on how to improve this with the SOF developers, and @marc-hb has been improving how we move forward here. We have a pending Readme patch to be merged before we tag today, i.e. the binaries are all correct but we will tag once the readme update is merged today.
perexg commentedon Jun 3, 2021
Ok, but how I can fetch only v1.8 with the current scheme? If you tag the main branch, I'll fetch both 1.7 and 1.8. And if there are 10 releases, I'll end to fetch all of them.
The solution may be to filter the files in the release assets, but they have wrong names (v1.7.tar.bz instead sof-bin-1.7.tar.gz) or return back to the release branches.
lgirdwood commentedon Jun 3, 2021
@marc-hb please also tag the release. You have maintainer access so should be able to do this via GH or git. The known issues are here https://github.com/thesofproject/sof/milestone/15 for linking to in release.
lgirdwood commentedon Jun 3, 2021
I think @marc-hb has some flows for installation. Lets wait until he's online later today.
marc-hb commentedon Jun 3, 2021
That's correct.
You can't do that anymore. The longer term fix for that is to move away from using git to release binaries. For now the entire
.git/folder is 10M.As you noted, tagging the releases in sof-bin does not really make sense anymore.
The 1.7 version in the README.md is obviously an example.
lgirdwood commentedon Jun 3, 2021
@marc-hb thanks for your inputs here, are there any docs in sof-docs or GH feature request that describe the release process roadmap and your current plans. I assume the end goal is to have a script that can create a tar.bz2 of FW, ldcs and topolgies for someone to attach to GH releases ?
perexg commentedon Jun 3, 2021
Just a note: From the storage perspective, it does not matter, if you store different source binary files to one branch or multiple branches. Perhaps, git merging may help, too.
I'm using the auto-generated tar balls by github now (using the commit-id not your tag which was floating previously). In this case, only the necessary files are downloaded. Your move to artifacts means that you plan to use own scripts for the release, which is also fine, but the storage can be used efficiently with the branches, too like:
In this case, you can just tag the version branch and you'll get the automatic tar ball from github.
Example of commit-id URL to get the tar ball from github for v1.7: https://github.com/thesofproject/sof-bin/archive/6bd5501e6ac3f29dde42a28ffe03a2942aa60640/sof-bin-6bd5501.tar.gz .
perexg commentedon Jun 3, 2021
Or you may create meta branch with scripts and doc files which will be merged to main (all branches) and to separate release branches. There are multiple possibilities.
marc-hb commentedon Jun 3, 2021
Most of this script already exists and I use it many times everyday: https://github.com/thesofproject/sof/tree/main/installer
It's still missing a few features: final tarball, checksums, compilation flags,... I was hoping to get some user feedback to help prioritize further development but I'm afraid most people still use xtensa-build-all.sh + build-tools.sh + their own, private scripts on top. Cause why change? This is also the case of our internal Jenkins-based build and validation; this mismatch makes each release time-consuming and error-prone.
It's been unsurprisingly hard to get people excited about packaging and releases so the only way forward has been slow: try something (not too) different and wait a bit to see if anyone is confused or complains. There :-)
Yes, my idea was to eventually just upload the output of the installer directly to https://docs.github.com/en/github/administering-a-repository/releasing-projects-on-github/managing-releases-in-a-repository and deprecate the entire sof-bin repo: everything required to perform releases would be self-contained in the main sof repo, no external dependencies and no other branches or tag besides the "real" sof ones. Hopefully simple enough not to require any Powerpoint: configure and run the
installer/, upload to github releases, done. No secret sauce hidden in Jenkins or wherever.Example: https://github.com/zephyrproject-rtos/sdk-ng/releases
First, thanks for this clarification and your expert feedback in general: even more useful considering how rare you are.
I didn't even know github had such a convenient
git archiveimplementation, sorry I "broke" that. I understand the convenience of tagging sof-bin now, I will do today.Once I tag sof-bin this git archive feature should not really be "broken" any more, just less efficient. Do you mind ignoring the older versions until we move to github releases? Just a few spurious megabytes every few months.
For the record I "announced" the move to a single branch in January 2021 (among many other things, some abandoned). I understand that didn't make any headline :-) thesofproject/sof#3775 (comment)
Cc: @aiChaoSONG , @keyonjie , @fredoh9 , @plbossart , @mrhpearson @cujomalainey , please forward to any other (rare!) person who might have an interest for this topic.
plbossart commentedon Jun 3, 2021
FWIW we added recently a pointer to sof-bin in the drivers:
you may change directions on how you want to deliver the binaries but that site has to stay.
marc-hb commentedon Jun 3, 2021
Good point thanks! We can have a simple "redirection" in the top-level sof-bin/README.md
As you made no other comment, I assume you agree with all the rest? :-)
plbossart commentedon Jun 3, 2021
As long as @perexg is happy I am happy.
README.md: remove some v1.7 hardcoding
README.md: remove some v1.7 hardcoding
1 remaining item
marc-hb commentedon Jun 3, 2021
Good idea and close enough to the final solution anyway; just in a different repo for now. Will do after README.md #37 is merged and after I create the v1.8-rc2 tag in sof-bin.
Thanks, I think I understand better now. In fact I just found https://koji.fedoraproject.org/koji/packageinfo?packageID=30869. I ran
dnf download --source alsa-sof-firmwareon some Fedora box and had a look at the.specand other files. I should have done this earlier.Are you expressing a space concerning here and/or other(s)?
Just to be extra sure: in this particular case, "source package" means "sof-bin sources" which means firmware binaries, correct?
perexg commentedon Jun 4, 2021
The Fedora package sources are available also via web portal: https://src.fedoraproject.org/rpms/alsa-sof-firmware
Here is a straight change from the commit-id to the version style for 1.7 (sitting in my private repo until we know the final source URL for 1.8):
I only mean that the source package contents propagate to all distributions which need to deal with it somehow. So removing duplicates is really welcome.
Yes, I mean the source for the packaging here (thus the sof-bin tar or zip archive) -
Source:line in the rpm spec file.Thank you for your attention and your work on this.
README.md: remove some v1.7 hardcoding
lgirdwood commentedon Jun 4, 2021
@mwasko who from your team can work with @marc-hb on aligning the signing process with the release process to make it less manual and error prone. I would imagine that some helper scripts could be used to take the signed output from the signing system and upload/copy to the release destination. This can be done for v1.9 so we have 3 months.
marc-hb commentedon Jun 4, 2021
@perexg can you try this when you get a chance? https://github.com/thesofproject/sof-bin/releases/tag/v1.8-rc2
perexg commentedon Jun 4, 2021
The paths are so complex, but it does work. Also, see the difference between v1.7 (easy) and v1.8-rc2 (complex).
marc-hb commentedon Jun 4, 2021
I just verified that I can change the "v1.8-rc2 only" tarball like this with changing neither the
.install.shnor theREADME.md:EDIT: automated and submitted in #48
So this should look exactly like v1.7 and should let you get back to a single
%{sof_version}variable, please confirm.I just updated the sof-bin-v1.8-rc2.tar.gz accordingly. Note: I normally never change anything released and I'm surprised github let me do that without even a warning. However this is a release candidate and it's a new approach that I uploaded less than one hour ago. So I'm pretty confident you've been the only user so far :-)
Add new tarball_one_version.sh to package a single version
perexg commentedon Jun 5, 2021
Much better now (it's diff from 1.7 release):
mwasko commentedon Jun 7, 2021
@slawblauciak and me are coordinating sof releases. I would recommend to prepare/extend script to validate release content and report if anything is missing. We can also work on some helper scripts to upload signed binaries, but there will be still some manual work dependency.
Add new tarball_one_version.sh to package a single version
marc-hb commentedon Jun 22, 2021
1.8 released at https://github.com/thesofproject/sof-bin/releases
@perexg is everything good or good enough now?
lgirdwood commentedon Jun 22, 2021
Thanks @marc-hb - Adding release notes shortly - give me a few hours.
perexg commentedon Jun 23, 2021
It seems fine. Fedora 34 package update: https://bodhi.fedoraproject.org/updates/FEDORA-2021-c838e6f856
I'm closing this now. It seems that all was settled.