Developing lightweight computation at the DSG edge

Skip to content

GitLab

Commit 9bb7c8af authored by Roger Pueyo Centelles's avatar Roger Pueyo Centelles
Browse files

Merge branch 'master' into testing

parents 94383f4e b0fb46a3
Hide whitespace changes
Inline Side-by-side
Dockerfile 0 → 100644
View file @ 9bb7c8af
FROM debian:latest
RUN apt-get update && apt-get install subversion build-essential libncurses5-dev zlib1g-dev gawk git ccache gettext libssl-dev xsltproc wget unzip python -y
WORKDIR /app
ENTRYPOINT ["/app/cooker"]
CMD ["--help"]
README.md
View file @ 9bb7c8af
......@@ -7,8 +7,17 @@ Basic usage example for cooking a firmware for TpLink 4300:
## Preparing the local environment
### Building in running system
Before using lime-sdk, make sure your Linux system has the required dependencies installed. You might follow these instructions (look for _Examples of Package Installations_) https://lede-project.org/docs/guide-developer/install-buildsystem
### Building in docker container
Install [Docker](https://www.docker.com/get-docker) and run the following command:
docker build -t cooker .
docker run -it cooker -v "$(pwd":/app --<parameters>
## Targets, profiles and flavors
LibreMesh can be used on many different devices (target and profile) and can be packed in many different ways (flavors), depending on your needs. To this end, it is important to choose the right options for building your firmware.
......@@ -82,6 +91,8 @@ Both community and device profile names can be any of your choice (must exist!)
`./cooker -c ar71xx/generic --profile=tl-wdr4300-v1 --flavor=lime_default --community=CommunityName/ProfileName`
A community profile might include a special file named PACKAGES on the root of the profile directory (_CommunityName/ProfileName/PACKAGES_) to specify a list of extra packages which must be added to the firmware image.
## Using development branch
If you want to get the last LEDE source because it includes some new feature or it supports some new hardware, you can use the lime-sdk branch named _develop_. However as LEDE source is changing daily, we cannot assure the correct working of the firmware.
......@@ -158,8 +169,8 @@ Time to time, if you want to update the code with the official one you might add
## Advanced help
Usage: ./cooker [-f [--force]] [-d <target> [--sdk|ib|force]] [-i <target> [--sdk-file=<file>|ib-file=<file>]]
[--download-all|build-all|update-feeds] [--targets|flavors|communities|profiles=<target>]
[-b <target> [--no-update|no-link-ib|remote|clean|package=<pkg>]]
[--download-all|build-all|update-feeds] [--targets|flavors|communities|update-communities|profiles=<target>]
[-b <target> [-j<N>] [--no-update|no-link-ib|remote|clean|force-local|package=<pkg>]]
[-c <target> [--profile=<profile>|no-update|remote|flavor=<flavor>|community=<path>|extra-pkg=<list>]]
[--help]
......@@ -171,6 +182,7 @@ Time to time, if you want to update the code with the official one you might add
--profiles=<target> : list available hardware profiles for a specific target
--flavors : list available LibreMesh flavors for cooking
--communities : list available community profiles
--update-communities : update or download community profiles
--update-feeds : update previously downloaded feeds (only works for Git feeds)
-f : download feeds based on feeds.conf.default file. Feeds will be shared among all targets
--force : force reinstall of feeds (remove old if exist)
......@@ -182,10 +194,12 @@ Time to time, if you want to update the code with the official one you might add
--sdk-file=<file> : specify SDK file to unpack
--ib-file=<file> : specify ImageBuilder file to unpack
-b <target> : build SDK for specific target and link it to the ImageBuilder
-j<N> : number of threads to use when building (recommended N=#cores+1)
--no-link-ib : do not download and link ImageBuilder when building the SDK
--no-update : do not update feeds when building SDK
--clean : clean sources before compiling
--package=<pkg_name> : build only a package (and its dependencies)
--force-local : force installation of all local SDK compiled packages when cooking firmware
-c <target> : cook the firmware for specific target. Can be used with next options
--profile=<profile> : use <profile> when cooking firmware (default is all available target profiles)
--flavor=<flavor> : use <flavor> when cooking firmware (default lime_default)
......
cooker
View file @ 9bb7c8af
......@@ -13,8 +13,8 @@ J=${J:-$make_j}
usage() {
echo "Usage: $0 [-f [--force]] [-d <target> [--sdk|ib|force]] [-i <target> [--sdk-file=<file>|ib-file=<file>]]
[--download-all|build-all|update-feeds] [--targets|flavors|communities|profiles=<target>]
[-b <target> [--no-update|no-link-ib|remote|clean|force-local|package=<pkg>]]
[--download-all|build-all|update-feeds] [--targets|flavors|communities|update-communities|profiles=<target>]
[-b <target> [-j<N>] [--no-update|no-link-ib|remote|clean|force-local|package=<pkg>]]
[-c <target> [--profile=<profile>|no-update|remote|flavor=<flavor>|community=<path>|extra-pkg=<list>]]
[--help]"
echo ""
......@@ -26,6 +26,7 @@ usage() {
echo " --profiles=<target> : list available hardware profiles for a specific target"
echo " --flavors : list available LibreMesh flavors for cooking"
echo " --communities : list available community profiles"
echo " --update-communities : update or download community profiles"
echo " --update-feeds : update previously downloaded feeds (only works for Git feeds)"
echo " -f : download feeds based on feeds.conf.default file. Feeds will be shared among all targets"
echo " --force : force reinstall of feeds (remove old if exist)"
......@@ -37,6 +38,7 @@ usage() {
echo " --sdk-file=<file> : specify SDK file to unpack"
echo " --ib-file=<file> : specify ImageBuilder file to unpack"
echo " -b <target> : build SDK for specific target and link it to the ImageBuilder"
echo " -j<N> : number of threads to use when building (recommended N=#cores+1)"
echo " --no-link-ib : do not download and link ImageBuilder when building the SDK"
echo " --no-update : do not update feeds when building SDK"
echo " --clean : clean sources before compiling"
......@@ -111,9 +113,24 @@ cook() {
echo "-> Cooking $target/$profile"
[ ! -d "$output_dir" ] && mkdir -p $output_dir
local ib_files=""
local myfiles="$PWD/$files_dir"
[ -n "$community_files" ] && myfiles="$community_files"
# handle community packages and files, and local files
local ib_files="${PWD}/$tmp_dir/files"
local myfiles="${PWD}/$files_dir"
rm -rf ${ib_files}
mkdir -p ${ib_files}
[ -n "$community_files" ] && {
[ -f "${community_files}/PACKAGES" ] && {
echo "--> Adding community packages: "
join_packages $(cat "${community_files}/PACKAGES")
}
echo "--> Copying community files"
cp -rf ${community_files}/* $ib_files/ 2>/dev/null
[ -f ${ib_files}/PACKAGES ] && rm -f ${ib_files}/PACKAGES
}
cp -rf ${myfiles}/* ${ib_files}/ 2>/dev/null
# make binary using imagebuilder
local fw_extra_name="$flavor"
[ -n "$community" ] && fw_extra_name="$community-$community_profile"
find $myfiles/* 2>/dev/null && ib_files="FILES=$myfiles" # if files directory is empty, IB crashes
......@@ -121,7 +138,10 @@ cook() {
[ -n "$profile" ] && profile="PROFILE=$profile"
[ -n "$no_update" ] && force_no_update="NO_UPDATE=1"
[ -f "files_remove" ] && files_remove="FILES_REMOVE=$(pwd)/files_remove"
make -C $ib image $profile PACKAGES="${!flavor} $PKG" EXTRA_IMAGE_NAME="$fw_extra_name" BIN_DIR="$output_dir" $ib_files $force_no_update $files_remove
echo "-> Cooking firmware image"
join_packages ${!flavor}
echo "--> Selected extra packages:$PKG"
make -C $ib image $profile PACKAGES="$PKG" EXTRA_IMAGE_NAME="$fw_extra_name" BIN_DIR="$output_dir" $ib_files $force_no_update $files_remove
ERROR=$?
if (($ERROR != 0)); then
echo -e "\n-> Firmware not cooked, something wrong happened on the ImageBuilder compilation process"
......@@ -232,8 +252,9 @@ build_pkg() {
echo "-> Building package $pkg on sdk $sdk"
make -j$J V=$V -C $sdk package/$pkg/{clean,compile,install} && {
echo "-> Build of $pkg successful"
local pkg_file=$(find $sdk/bin/packages -type f -name "${pkg}*.ipk")
cp -f $pkg_file $output_dir/
local pkg_path=$(find $sdk/bin/packages -type f -name "${pkg}*.ipk")
local pkg_file=$(echo $pkg_path | awk -F/ '{print $NF}')
cp -f $pkg_path $output_dir/
echo "-> You can find it on $output_dir/$pkg_file"
} || echo "-> Build error, see full output using V=s J=1 ./cooker <options>"
}
......@@ -251,7 +272,7 @@ download_feeds() {
[ ! -d $feeds_dir ] && mkdir -p "$feeds_dir"
echo "-> Using feeds from file $feeds_template"
echo "-> Downloading feeds into $feeds_dir/"
set -e
while read feed; do
echo "$feed" | grep -q ^src-git || continue
name="$(echo $feed | awk '{print $2}')"
......@@ -269,6 +290,7 @@ download_feeds() {
fi
echo "src-link $name $PWD/$feeds_dir/$name" | tee -a $feeds_file
done < $feeds_template
set +e
}
download_communities() {
......@@ -281,6 +303,15 @@ download_communities() {
}
}
update_communities() {
[ ! -d "$communities_dir" ] && download_communities || {
(cd $communities_dir && git pull) || {
echo "-> Cannot update communities repository"
exit 1
}
}
}
list_communities() {
download_communities
find $communities_dir/ -maxdepth 1 -type d | grep -v .git | \
......@@ -376,7 +407,7 @@ unpack_ib() {
}
list_flavors() {
cat flavors.conf | egrep -v "^_|^#|^$" | awk -F= '{print $1}'
cat qmp.flavors.conf | egrep -v "^_|^#|^$" | awk -F= '{print $1}'
}
get_profile_names() {
......@@ -398,10 +429,18 @@ list_profiles() {
make -C $release/$target/ib info
}
# join a list of new packages into PKG var, older has preference
join_packages() {
local new="$@"
for p in $new; do
echo "$PKG" | egrep -q "^(|-)${p#-*}($|)" || PKG="$PKG $p"
done
}
[ -z "$1" ] && usage
OPTS=$(getopt -o hd:fb:c:d:i: -l targets,build-all,download-all,profiles:,flavors,profile:,flavor:,\
OPTS=$(getopt -o hd:fb:c:d:i:j: -l targets,build-all,download-all,profiles:,flavors,profile:,flavor:,\
update-feeds,no-update,no-link-ib,only-sdk,only-ib,remote,ib,sdk,sdk-file:,ib-file:,help,force,clean,\
communities,community:,cook-all,package:,extra-pkg:,force-local -n $0 -- "$@")
communities,update-communities,community:,cook-all,package:,extra-pkg:,force-local -n $0 -- "$@")
eval set -- "$OPTS"
while true; do
......@@ -490,13 +529,17 @@ while true; do
target=$2
shift;shift
;;
-j)
J=$2
shift;shift
;;
--clean)
sdk_clean=1
shift
sdk_clean=1
shift
;;
--package)
package=$2
shift;shift
package=$2
shift;shift
;;
-c)
cook=1
......@@ -519,13 +562,19 @@ while true; do
force_local="1"
shift
;;
--update-communities)
update_communities="1"
shift
;;
--community)
community="${2%/*}"
community_profile="${2#*/}"
shift;shift
;;
--extra-pkg)
system_pkg="$PKG"
PKG="$2"
join_packages $system_pkg
shift;shift
;;
-h)
......@@ -563,19 +612,21 @@ profile=${profile:-Default}
download_feeds
}
[ "$update_communities" == "1" ] && update_communities
[ "$build_sdk" == "1" ] && build_sdk $target
[ "$cook" == "1" ] && {
[ -n "$community" ] && [ -z "$community_profile" ] && {
echo "-> If community is defined you must also specify its profile: --community=name/profile"
exit 0
exit 1
} || [ -n "$community" ] && {
download_communities
community_files="$PWD/$communities_dir/$community/$community_profile"
[ ! -d "$community_files" ] && {
echo "-> The community and profile specified does not exist ($community_files)"
exit 0
} || echo "-> Using Community files from $community_files"
exit 1
} || echo "-> Using community files from $community_files"
}
if [ "$profile" == "Default" ]; then
cook_all_profiles $target
......
feeds.conf.default
View file @ 9bb7c8af
......@@ -3,7 +3,3 @@ src-git packages https://git.lede-project.org/feed/packages.git;lede-17.01
src-git luci https://git.lede-project.org/project/luci.git;lede-17.01
src-git routing https://git.lede-project.org/feed/routing.git;lede-17.01
src-git telephony https://git.lede-project.org/feed/telephony.git;lede-17.01
src-git libremesh https://github.com/libremesh/lime-packages.git;17.06
src-git libremap https://github.com/libremap/libremap-agent-openwrt.git;master
src-git limeui https://github.com/libremesh/lime-ui-ng.git;master
targets.list
View file @ 9bb7c8af
ath25/generic
ar71xx/generic
ar71xx/mikrotik
ath25/generic
brcm2708/bcm2708
brcm2708/bcm2709
brcm2708/bcm2710
mpc85xx/generic
ramips/rt305x
ramips/rt3883
ramips/mt7620
ramips/mt7621
ramips/mt7628
ramips/mt7688
sunxi/generic
ramips/mt76x8
sunxi/cortexa53
sunxi/cortexa7
sunxi/cortexa8
x86/64
x86/generic
x86/geode
x86/64
x86/legacy
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment