[Qemu-discuss] Chroot exec error inspite of qemu-static-arm being installed

[Urmish Ajit Thakker]

Hi,

I am trying to chroot into an arm system from a x86 host. However, I
keep getting the following error

chroot: failed to run command `/bin/bash': Exec format error.

I think I have already installed the required files and I am not able to
debug the issue. Can anyone help me with this?

_Some more info_ -
Ubuntu - 12.04
Image - ARMv8 image from Linaro
When I run sudo update-binfmts --display

My output is -

/qemu-microblaze (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\xba\xab
mask =
\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff
interpreter = /usr/bin/qemu-microblaze-static
detector =
qemu-arm (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00
mask =
\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff
interpreter = /usr/bin/qemu-arm-static
detector =
qemu-m68k (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x04
mask =
\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff
interpreter = /usr/bin/qemu-m68k-static
detector =
qemu-ppc64abi32 (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x15
mask =
\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff
interpreter = /usr/bin/qemu-ppc64abi32-static
detector =
qemu-sparc (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x02
mask =
\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff
interpreter = /usr/bin/qemu-sparc-static
detector =
qemu-sparc64 (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x02\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x2b
mask =
\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff
interpreter = /usr/bin/qemu-sparc64-static
detector =
qemu-sh4 (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x2a\x00
mask =
\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff
interpreter = /usr/bin/qemu-sh4-static
detector =
qemu-sh4eb (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x2a
mask =
\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff
interpreter = /usr/bin/qemu-sh4eb-static
detector =
python2.7 (enabled):
package = python2.7
type = magic
offset = 0
magic = \x03\xf3\x0d\x0a
mask =
interpreter = /usr/bin/python2.7
detector =
qemu-sparc32plus (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x12
mask =
\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff
interpreter = /usr/bin/qemu-sparc32plus-static
detector =
qemu-ppc64 (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x02\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x15
mask =
\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff
interpreter = /usr/bin/qemu-ppc64-static
detector =
cli (enabled):
package = mono-runtime
type = magic
offset = 0
magic = MZ
mask =
interpreter = /usr/bin/cli
detector = /usr/lib/cli/binfmt-detector-cli
jar (enabled):
package = openjdk-6
type = magic
offset = 0
magic = PK\x03\x04
mask =
interpreter = /usr/bin/jexec
detector =
qemu-mipsel (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08\x00
mask =
\xff\xff\xff\xff\xff\xff\xff\x00\xfe\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff
interpreter = /usr/bin/qemu-mipsel-static
detector =
qemu-ppc (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x14
mask =
\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff
interpreter = /usr/bin/qemu-ppc-static
detector =
qemu-alpha (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x26\x90
mask =
\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff
interpreter = /usr/bin/qemu-alpha-static
detector =
jarwrapper (enabled):
package = <local>
type = magic
offset = 0
magic = PK\x03\x04
mask =
interpreter = /usr/bin/jarwrapper
detector = /usr/bin/jardetector
qemu-cris (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x4c\x00
mask =
\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff
interpreter = /usr/bin/qemu-cris-static
detector =
qemu-mips (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08
mask =
\xff\xff\xff\xff\xff\xff\xff\x00\xfe\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff
interpreter = /usr/bin/qemu-mips-static
detector =
python3.2 (enabled):
package = python3.2
type = magic
offset = 0
magic = \x6c\x0c\x0d\x0a
mask =
interpreter = /usr/bin/python3.2
detector =
qemu-armeb (enabled):
package = qemu-user-static
type = magic
offset = 0
magic =
\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28
mask =
\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff
interpreter = /usr/bin/qemu-armeb-static
detector = /

[Peter Maydell]

Hi,
I am trying to chroot into an arm system from a x86 host. However, I keep
getting the following error
chroot: failed to run command `/bin/bash': Exec format error.
I think I have already installed the required files and I am not able to
debug the issue. Can anyone help me with this?

A set of things you can try to narrow down the problem:

/path/to/chroot/usr/bin/qemu-arm-static --help
# if this doesn't work you haven't installed the right QEMU into
# your chroot
sudo chroot /path/to/chroot /usr/bin/qemu-arm-static --help
# if this doesn't work then your allegedly static QEMU
# isn't actually statically linked
/path/to/chroot/usr/bin/qemu-arm-static /path/to/chroot/bin/ls
# this *should* fail with "/lib/ld-linux.so.3: No such file or directory"
# or similar, indicating we read the executable but couldn't find
# the dynamic linker. If this fails then maybe your chroot doesn't
# have ARM binaries in it...
sudo chroot /path/to/chroot /usr/bin/qemu-arm-static /bin/ls
# this should work and list the files in the chroot directory.
# if it doesn't work then likely your chroot is wrongly set up
sudo chroot /path/to/chroot /bin/ls
# if this fails but the previous command worked, then your binfmt-misc
# registration is broken.

thanks
-- PMM

[Urmish Ajit Thakker]

Hi,

Thanks for the reply.
I tried your advice and got the following results.
I cd into my chroot directory -
cd <mount place of image>

1) usr/bin/qemu-arm-static --help
R - Prints out help

2) sudo chroot . /usr/bin/qemu-arm-static --help
R - Prints out help

3) usr/bin/qemu-arm-static bin/ls
E - Error while loading bin/ls: No such file or directory

Note : bin/ls exists

4) sudo chroot . /usr/bin/qemu-arm-static /bin/ls
R - /bin/ls: Invalid ELF image for this architecture

5) sudo chroot . /bin/ls
R - chroot: failed to run command `/bin/ls': Exec format error

If I understand your instructions clearly, 3 & 4 should not happen. Am I
correct?

Regards,
Urmish

Post by Peter Maydell

Hi,
I am trying to chroot into an arm system from a x86 host. However, I keep
getting the following error
chroot: failed to run command `/bin/bash': Exec format error.
I think I have already installed the required files and I am not able to
debug the issue. Can anyone help me with this?

/path/to/chroot/usr/bin/qemu-arm-static --help
# if this doesn't work you haven't installed the right QEMU into
# your chroot
sudo chroot /path/to/chroot /usr/bin/qemu-arm-static --help
# if this doesn't work then your allegedly static QEMU
# isn't actually statically linked
/path/to/chroot/usr/bin/qemu-arm-static /path/to/chroot/bin/ls
# this *should* fail with "/lib/ld-linux.so.3: No such file or directory"
# or similar, indicating we read the executable but couldn't find
# the dynamic linker. If this fails then maybe your chroot doesn't
# have ARM binaries in it...
sudo chroot /path/to/chroot /usr/bin/qemu-arm-static /bin/ls
# this should work and list the files in the chroot directory.
# if it doesn't work then likely your chroot is wrongly set up
sudo chroot /path/to/chroot /bin/ls
# if this fails but the previous command worked, then your binfmt-misc
# registration is broken.
thanks
-- PMM

[Peter Maydell]

Post by Urmish Ajit Thakker
3) usr/bin/qemu-arm-static bin/ls
E - Error while loading bin/ls: No such file or directory
Note : bin/ls exists
4) sudo chroot . /usr/bin/qemu-arm-static /bin/ls
R - /bin/ls: Invalid ELF image for this architecture
5) sudo chroot . /bin/ls
R - chroot: failed to run command `/bin/ls': Exec format error
If I understand your instructions clearly, 3 & 4 should not happen. Am I
correct?

Correct, and this also shows that your problem is not related
to the binfmt-misc registration. It looks like your chroot does
not contain valid 32 bit ARM binaries. What does
file /path/to/chroot/bin/ls
say? (if it says it's a symlink or something then find
a binary in the chroot that's a real target executable).

-- PMM

[Urmish Ajit Thakker]

Some more info -

I am running an Ubuntu 12.04 machine.

Disk Image is for armv8 architecture. (64 bit).

So my ls -l bin/ls gives me the following outout

bin/ls -> /bin/busybox.nosuid
And bin/busybox.nosuid exists in the mount directory.

I feel my qemu is old. But Ubuntu 12.04 does not have the latest package
of qemu which can mount armv8. Does this make any sense?

Regards,
Urmish

Post by Peter Maydell

Post by Urmish Ajit Thakker
3) usr/bin/qemu-arm-static bin/ls
E - Error while loading bin/ls: No such file or directory
Note : bin/ls exists
4) sudo chroot . /usr/bin/qemu-arm-static /bin/ls
R - /bin/ls: Invalid ELF image for this architecture
5) sudo chroot . /bin/ls
R - chroot: failed to run command `/bin/ls': Exec format error
If I understand your instructions clearly, 3 & 4 should not happen. Am I
correct?

Correct, and this also shows that your problem is not related
to the binfmt-misc registration. It looks like your chroot does
not contain valid 32 bit ARM binaries. What does
file /path/to/chroot/bin/ls
say? (if it says it's a symlink or something then find
a binary in the chroot that's a real target executable).
-- PMM

[Peter Maydell]

Post by Urmish Ajit Thakker
Some more info -
I am running an Ubuntu 12.04 machine.
Disk Image is for armv8 architecture. (64 bit).

Ah, I had wondered. qemu-arm-static is for 32 bit ARM.
For 64 bit you want qemu-aarch64-static...

Post by Urmish Ajit Thakker
So my ls -l bin/ls gives me the following outout
bin/ls -> /bin/busybox.nosuid
And bin/busybox.nosuid exists in the mount directory.

...and if you run 'file' on that...?

thanks
-- PMM

[Urmish Ajit Thakker]

...and if you run 'file' on that...?

This is the file result .... in/busybox.nosuid: ELF 64-bit LSB executable, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 3.7.0, BuildID[sha1]=0x449d66054f43dcae8f28ccd92e17c0db27455215, stripped

and ldd bin/busybox.nosuid
not a dynamic executable

So I upgraded my ubuntu to 14.04 and installed qemu-aarch64-static. Now
if I do chroot it tells me "cannot find" /bin/bash! Although it is there!

Wierd thing is, if I run an older image of arm (32 bit architecture). I
am able to modify the image.

Regards,
Urmish

Post by Peter Maydell

Post by Urmish Ajit Thakker
Some more info -
I am running an Ubuntu 12.04 machine.
Disk Image is for armv8 architecture. (64 bit).

Ah, I had wondered. qemu-arm-static is for 32 bit ARM.
For 64 bit you want qemu-aarch64-static...

Post by Urmish Ajit Thakker
So my ls -l bin/ls gives me the following outout
bin/ls -> /bin/busybox.nosuid
And bin/busybox.nosuid exists in the mount directory.

...and if you run 'file' on that...?
thanks
-- PMM

[Peter Maydell]

Post by Peter Maydell
...and if you run 'file' on that...?
This is the file result .... in/busybox.nosuid: ELF 64-bit LSB executable,
version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux
3.7.0, BuildID[sha1]=0x449d66054f43dcae8f28ccd92e17c0db27455215, stripped
and ldd bin/busybox.nosuid
not a dynamic executable
So I upgraded my ubuntu to 14.04 and installed qemu-aarch64-static. Now if I
do chroot it tells me "cannot find" /bin/bash! Although it is there!

...again, run through all the steps I suggested earlier to
narrow down what exactly is the problem.

-- PMM

[Urmish Ajit Thakker]

Here are the results for the Ubuntu 14.04 and 64 bit arm

The chroot runs for the latest arm image now. I had accidentally copied
qemu-arm instead of qemu-aarch64 for the 64 bit arm image.

Thanks for the help!

Regards,
Urmish

Post by Peter Maydell

Post by Peter Maydell
...and if you run 'file' on that...?
This is the file result .... in/busybox.nosuid: ELF 64-bit LSB executable,
version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux
3.7.0, BuildID[sha1]=0x449d66054f43dcae8f28ccd92e17c0db27455215, stripped
and ldd bin/busybox.nosuid
not a dynamic executable
So I upgraded my ubuntu to 14.04 and installed qemu-aarch64-static. Now if I
do chroot it tells me "cannot find" /bin/bash! Although it is there!

...again, run through all the steps I suggested earlier to
narrow down what exactly is the problem.
-- PMM