Third party cookies may be stored when visiting this site. Please see the cookie information.

Penguin Fortress YouTube Channel

Video transcript: Hacking the Linux passwd and shadow files

This is a transcript for a video linked here: Hacking the Linux passwd and shadow files.

Video transcript - Hacking the Linux passwd and shadow files

00:00:00.480 --> 00:00:05.840

hello this video is a guide to how

linux stores usernames and passwords

00:00:06.640 --> 00:00:11.520

this will include instructions on how you can

perform password recovery on a linux system

00:00:12.240 --> 00:00:17.840

using raspberry pi as an example and we're

going to show how you can reset the password

00:00:17.840 --> 00:00:23.200

without knowing it in the first place this

video is also an introduction for a video

00:00:23.200 --> 00:00:29.840

i plan to make in the future which will show

how hackers try and crack the system passwords

00:00:31.440 --> 00:00:35.440

in my earlier video i explained about the

importance of passwords in this video i'm

00:00:35.440 --> 00:00:40.080

going to look at these in a bit more depth

on how they are stored on the linux system

00:00:41.840 --> 00:00:47.360

and then the future video will show some of the

vulnerabilities of how they are stored i'm going

00:00:47.360 --> 00:00:52.320

to start with a little history just because this

will make the explanation both little simpler

00:00:52.320 --> 00:00:58.960

and it'll explain the reason that the password

field is configured the way it is linux is based

00:00:58.960 --> 00:01:05.600

on the unix operating system back in the 1980s and

earlier as my children would say the olden days

00:01:06.240 --> 00:01:11.840

the password information was

stored in the password file

00:01:12.960 --> 00:01:17.360

the file is often known as the

etc password or passwd file

00:01:17.920 --> 00:01:25.360

and it's available to anybody on the local

system to be able to read this has some

00:01:25.360 --> 00:01:32.640

security implications which i'll explain about

later the file is plain text so anybody can view

00:01:32.640 --> 00:01:39.120

it although you need to be root to be able to

edit it and just go through the fields here the

00:01:39.680 --> 00:01:46.160

top line shows the names of the fields and now

i'm going to show a working example here so

00:01:46.160 --> 00:01:50.960

the first field is the username typically

this represents the real name of the user

00:01:52.320 --> 00:01:56.640

or it may refer to a system account

such as root which is the admin account

00:01:59.760 --> 00:02:05.840

the next field is the password if we put the plain

text password here then it would be readable for

00:02:05.840 --> 00:02:13.040

anybody which is obviously bad instead this is

a hash value of the password and this particular

00:02:13.040 --> 00:02:20.160

example is an md5 hash of the word password i'll

explain hashing and its vulnerabilities later in

00:02:20.160 --> 00:02:27.280

the video on newer systems the password is moved

to the shadow file and instead you just get an x

00:02:27.280 --> 00:02:33.840

in this position in the password file this has

an extra level of security and i'll explain all

00:02:33.840 --> 00:02:41.040

about why this came about you may also see an

asterisk or an exclamation mark which indicate

00:02:41.040 --> 00:02:46.560

that the account will not accept logins although

typically that will be in the shadow file as well

00:02:47.840 --> 00:02:52.880

but if the password field is blank then a user

can log in without needing supply password

00:02:53.520 --> 00:02:58.560

which is usually a bad idea but useful

for password recovery as i'll show later

00:03:01.680 --> 00:03:09.520

next are the user id and group id the uid is a

unique number for this particular user and the gid

00:03:09.520 --> 00:03:16.720

is id of the default group for this user on modern

systems they'll normally be a group name which

00:03:16.720 --> 00:03:22.640

is the same as a username and these are created

automatically when you use the add user command

00:03:23.600 --> 00:03:31.680

and the uid and gid will be the same for user

accounts the uid and gid typically starts at 1000

00:03:33.120 --> 00:03:40.400

the lower numbers are reserved for the special

accounts now here that i'm showing that line

00:03:40.400 --> 00:03:49.280

wrap into the next line this would actually all

be on a single line the next field is shown is

00:03:49.280 --> 00:03:59.520

known as the gcos field i'll refer to above as

the user info field and this is where you include

00:03:59.520 --> 00:04:05.040

information about the user such as their name

possibly their phone number or other information

00:04:08.480 --> 00:04:13.360

next is home directory normally each

user has their own home directory which

00:04:13.360 --> 00:04:17.440

is in the slash home directory and

is named the same as their username

00:04:20.400 --> 00:04:25.680

and the final field is the shell this is the

command line user interface that you get when

00:04:25.680 --> 00:04:34.480

logging in through a text terminal or using a

remote terminal such as ssh for most users this

00:04:34.480 --> 00:04:41.280

will be the best shell which is slash bin bash for

system accounts that do not allow logins then this

00:04:41.280 --> 00:04:48.720

is normally set to slash usr slash s bin no login

which will not allow interactive shell sessions

00:04:51.600 --> 00:05:00.560

as i mentioned previously this is the password

field and you can't put the plain text password

00:05:00.560 --> 00:05:04.800

in that field as it obviously everyone then would

be able to read it and know what your password was

00:05:05.840 --> 00:05:11.120

so we need a way that the computer can know

if you've entered the correct password but so

00:05:11.120 --> 00:05:16.640

that anyone else that looks at it can't tell what

your password is this is where a hashing algorithm

00:05:16.640 --> 00:05:24.640

comes in i'm going to show this using md5 this was

used for passwords in the past but the algorithms

00:05:24.640 --> 00:05:30.080

used today are usually a lot more secure md5

is still quite widely used for other purposes

00:05:33.040 --> 00:05:41.040

the md5 stands for the message digest algorithm an

md5 is one implementation it's not necessarily the

00:05:41.040 --> 00:05:46.720

fifth version as the name might suggest because

some of the version numbers were never actually

00:05:46.720 --> 00:05:55.200

used i'm going to walk through how the md5 hashing

algorithm works we start with the password here

00:05:56.480 --> 00:06:01.120

and here i've used the word password obviously

you'd use a more secure password than this

00:06:03.440 --> 00:06:11.680

this is processed by the md5 hashing algorithm

and this results in a 32 character long string

00:06:12.640 --> 00:06:17.600

note that this is actually a 16 bit

hexadecimal number that represented as text

00:06:18.960 --> 00:06:22.560

the important thing is the direction

of the arrows in the diagram

00:06:23.840 --> 00:06:29.200

so now that whilst you can go from the

password and get the output it is not

00:06:29.200 --> 00:06:34.320

possible to reverse this and get back

to the original password if you have the

00:06:35.520 --> 00:06:42.720

hashed output if that was possible it would

break the whole concept of the password hash

00:06:43.280 --> 00:06:52.000

and it would render it unsafe i won't go into the

details of how the hashing algorithm works this

00:06:52.000 --> 00:06:59.040

diagram gives an approximation the important thing

is not to be able to go in the reverse direction

00:07:00.160 --> 00:07:06.640

now this is one of the key differences between

encryption which can be reversed versus hashing

00:07:06.640 --> 00:07:13.040

which can't obviously for encryption you do need

to know the key and the algorithm to be able to

00:07:13.040 --> 00:07:21.040

reverse it but even if you know how the algorithm

works for a hashing algorithm you can't reverse it

00:07:23.360 --> 00:07:28.560

so only the output of the hash is ever saved on

the computer and after storing it in the password

00:07:28.560 --> 00:07:36.320

file the computer cannot find out what your

original password was as it cannot be reversed the

00:07:36.320 --> 00:07:41.520

way the computer is able to validate the password

is that when you enter your password it performs

00:07:41.520 --> 00:07:47.600

the same hash algorithm against it and checks

to see if it matches the saved password hash

00:07:49.520 --> 00:07:54.960

now md5 is now considered to be broken it's not

that it's possible to go in the reverse direction

00:07:56.240 --> 00:08:01.600

but it may be possible to create a collision

instead which is another string that gives the

00:08:01.600 --> 00:08:06.960

same md5 hashtag value so whilst you might not

be able to get back to the original password

00:08:06.960 --> 00:08:12.160

what you might be able to do is think of another

password that would give you the same value

00:08:13.520 --> 00:08:18.000

it's still quite complicated

but it is theoretically possible

00:08:21.040 --> 00:08:25.520

even before the algorithm was broken there were

other ways of defeating the clashing algorithm

00:08:25.520 --> 00:08:31.280

method of creating passwords and some of these are

still possible with some of the other algorithms

00:08:31.280 --> 00:08:36.400

i've discussed some of these in my earlier

video where i discussed ways of guessing the

00:08:36.400 --> 00:08:42.640

password using dictionary attacks and brute force

attacks and these both rely on trying different

00:08:42.640 --> 00:08:47.200

combinations of possible passwords and just

passing them through the same hashing algorithm

00:08:50.080 --> 00:08:56.720

an additional risk with a hashing algorithm

such as md5 is that it's possible to create

00:08:56.720 --> 00:09:04.640

rainbow tables these are often used by online

md5 decryption tools and you're able to have a

00:09:04.640 --> 00:09:11.920

go one of those if you want they're a form of

brute force attack but instead of needing to

00:09:12.560 --> 00:09:19.120

re run the algorithm for every permutation

they run once and compute it into a large table

00:09:20.000 --> 00:09:23.040

and then that table can be

efficiently searched afterwards

00:09:25.200 --> 00:09:30.880

a way of defeating this is by adding a salt to

the password and this salt is a known string and

00:09:30.880 --> 00:09:35.920

it's normally added to the start of the password

before the hashing and it prevents the tables

00:09:35.920 --> 00:09:43.520

being used so as long as different salts are used

for different systems it effectively makes it

00:09:43.520 --> 00:09:50.880

unfeasible to be able to create sufficient rainbow

tables to be able to break it using this method

00:09:52.800 --> 00:10:01.920

there's a few of the hashing algorithms that are

used for linux passwords and when you look at the

00:10:01.920 --> 00:10:11.360

password string that is generated they start

with a few characters and these are telling

00:10:11.360 --> 00:10:18.720

you the algorithm that's used so that the system

knows what algorithm to apply to your password

00:10:21.600 --> 00:10:26.640

and this choice is normally made by your

distribution but if you have a choice of

00:10:26.640 --> 00:10:34.720

hashing algorithm then i would recommend the

two bottom ones are the sha 512 or yes crypt

00:10:37.360 --> 00:10:43.520

and those are the ones that are commonly used on

most linux systems so if you look at this it's

00:10:43.520 --> 00:10:50.320

showing you that md5 the first few letters of

the password entry would say dollar one dollar

00:10:51.280 --> 00:11:00.080

and for yes crypt it's dollar y dollar and

you'll be able to see those if you look in your

00:11:00.080 --> 00:11:05.760

password file or technically your shadow file

will come on to that in a minute so although

00:11:05.760 --> 00:11:12.560

the new hashing algorithms are much safer they

have still got risk of dictionary or brute force

00:11:12.560 --> 00:11:18.800

attacks especially if the user chooses a poor

password and this is why the password is no longer

00:11:18.800 --> 00:11:28.640

stored in the etc password file instead it's

moved into a different file called /etc/shadow

00:11:30.320 --> 00:11:34.720

which is a bit of a compromise because the

ideal solution would be to make the /etc/passwd

00:11:34.720 --> 00:11:40.800

file so that only the admin root user could read

it and then you can just have it all in one file

00:11:41.520 --> 00:11:48.080

however that would have broken other programs

which were relying on being able to access the

00:11:48.880 --> 00:11:53.920

safer information that's contained in that

file so the solution they came up with was

00:11:53.920 --> 00:12:04.080

to remove the password from etc passwd file and

instead store that into the file etc shadow and

00:12:04.080 --> 00:12:11.200

if you look at the passwd file the password field

has been replaced by an x lowercase x character

00:12:11.200 --> 00:12:18.320

which tells the system to look at the shadow file

instead of the password being in the password file

00:12:19.120 --> 00:12:24.480

this shows the format of the shadow file and the

username is the same as the password file and

00:12:24.480 --> 00:12:30.800

then the password which is stored in the second

field and again this is the hashed password field

00:12:32.160 --> 00:12:36.400

they also added other fields which

store details about when the password

00:12:36.400 --> 00:12:41.200

was last changed the minimum and maximum

days that the password is valid for

00:12:42.400 --> 00:12:46.080

the number of days to warn

before forcing a password change

00:12:47.120 --> 00:12:54.480

the number of days after it's inactive before it

gets disabled and finally when the account expires

00:12:57.920 --> 00:13:02.800

as promised i'm now going to show how you

can use what we've just learned about the

00:13:02.800 --> 00:13:07.280

password and shadow files if you find

yourself locked out of a linux system

00:13:08.480 --> 00:13:14.560

this is based on using a raspberry pi it'll also

work with other linux systems but in that case

00:13:14.560 --> 00:13:20.960

you will either need to boot into a system remote

or what's known as recovery mode or perhaps use

00:13:20.960 --> 00:13:29.520

a live boot disk to be able to access the system

and then onto the drives alternatively you could

00:13:29.520 --> 00:13:36.480

always remove the drives from the computer

but that's of a bit more hands-on physically

00:13:36.480 --> 00:13:42.000

with a computer compared to the raspberry pi where

it's just a case of pulling out your micro sd card

00:13:42.800 --> 00:13:52.000

here i am i've got a raspberry pi system

and i've set a very long password and

00:13:53.040 --> 00:14:00.080

i don't know what the password is so effectively

i'm logged in i can try guessing a password

00:14:01.600 --> 00:14:10.160

i know that the user is pixel server but if i try

the password hit enter i'll click log in incorrect

00:14:10.160 --> 00:14:17.760

password please try again and without knowing

that password obviously i can't get on the system

00:14:19.760 --> 00:14:27.760

however the one thing about linux systems in

fact most computer systems if you can physically

00:14:27.760 --> 00:14:34.800

access the drive and the data is not encrypted

then there's ways of resetting the password

00:14:34.800 --> 00:14:41.440

so that's what we're going to do i'm going to shut

this computer down and then remove the sd card

00:14:42.480 --> 00:14:50.080

bring that into another computer and show

you how i can basically remove the passwords

00:14:52.320 --> 00:14:59.040

so i've taken the sd card out of the raspberry pi

i'm going to pop this onto my linux computer here

00:14:59.920 --> 00:15:09.520

so you need to put this into a computer that's

able to read ext4 file systems which is what is

00:15:09.520 --> 00:15:20.320

used by linux so as you can see the elements is

is my external drive so we can ignore that we've

00:15:20.320 --> 00:15:28.320

got two drives appeared one's boot and the other

is called rootfs boot is the fat32 file system

00:15:28.320 --> 00:15:35.440

that holds the boot information but the root fs

is the entire root file system on the raspberry

00:15:35.440 --> 00:15:42.240

pi so this is the one we're interested

in and we're going to click on that and

00:15:42.240 --> 00:15:48.080

we're going to go into etc and the password

file and the shadow files are both in here

00:15:51.600 --> 00:15:59.840

there we are we've got the password and the

shadow files you'll see that we're actually

00:15:59.840 --> 00:16:06.320

unable to edit this at the moment and we don't

have permissions to view that that's what this red

00:16:06.320 --> 00:16:13.600

circle is so we're gonna have to do this as root i

can do is launch terminal and then we'll change to

00:16:13.600 --> 00:16:23.120

that so it's it's mounted into media then under

my own username and then it's called root fs

00:16:27.520 --> 00:16:36.160

and it's then in the etc and it's the shadow so

you have to use sudo to edit this as root and we

00:16:36.160 --> 00:16:43.680

can edit i'll use the via editor you could use

nano or you could use any other editor you want

00:16:47.440 --> 00:16:53.920

and now i'm going to go through

this to find the password of the

00:16:53.920 --> 00:16:58.960

user that i wanted to get on so you can

see says pixel server is the username

00:16:59.600 --> 00:17:05.760

and as i said the dollar y

dollar indicates that this is a

00:17:08.160 --> 00:17:20.320

yes crypt a hash value and basically we can just

remove that entire thing up to the next colon

00:17:21.760 --> 00:17:26.160

and now this has got no password and that

means you can just log in without any password

00:17:28.160 --> 00:17:34.000

we'll save that i'm going to just run sync that's

just going to make sure that it's written to the

00:17:35.440 --> 00:17:43.440

sd card and it's not just kept in a buffer

anywhere and then i can switch back to the

00:17:44.560 --> 00:17:52.800

window we can just eject that busy because

i'm still in it so just eject that now

00:17:54.720 --> 00:18:01.920

it said it can now be safe to remove so i'll

pop that out pop that back into the raspberry pi

00:18:01.920 --> 00:18:09.200

and we'll see if that's worked so i've taken that

sd card after i've modified the shadow file and

00:18:09.200 --> 00:18:17.280

put that back into the raspberry pi and booted

up now without a password in find a lot of the

00:18:17.920 --> 00:18:22.480

default network services won't allow you to log

in anyway they'll only let you log in if there

00:18:22.480 --> 00:18:31.360

is actually a password set so in this case i'm

actually physically on the raspberry pi here

00:18:32.080 --> 00:18:42.560

i've got a console connected directly to the

hdmi output so i can log in directly here

00:18:44.080 --> 00:18:51.440

the username's pixel server just leave password

blank and click login and as you can see it's

00:18:51.440 --> 00:18:58.000

taking me straight in there's no password needed

and that's thanks to editing that password file

00:18:58.880 --> 00:19:01.440

very important that you change

that password straight away

00:19:02.720 --> 00:19:05.280

so we can just use the passwd command

00:19:07.360 --> 00:19:10.160

now i'm going to use a really simple password here

00:19:12.160 --> 00:19:15.440

just going to use the word password

obviously you wouldn't use that

00:19:18.560 --> 00:19:23.840

but this will now allow me to

have a look at that shadow file

00:19:28.880 --> 00:19:30.080

that we edited earlier

00:19:32.160 --> 00:19:35.520

and you'll see that this has

put that password in so this

00:19:35.520 --> 00:19:40.320

is the yes crypt version of the word password

00:19:43.680 --> 00:19:50.240

and that's how you can recover a server

where you've logged and lost the password

00:19:50.240 --> 00:19:54.800

the important thing really is that it's the

physical security that prevents you doing this

00:19:55.360 --> 00:20:02.720

if you can physically access a device then

you're going to be able to get to the data

00:20:02.720 --> 00:20:06.720

unless it's all encrypted and that's

the only way you can really protect

00:20:07.280 --> 00:20:12.640

the data if somebody can physically get

hold of the disks that it's stored on

00:20:14.560 --> 00:20:18.000

so obviously i'm going to change this

password now to a nice secure one

00:20:19.760 --> 00:20:24.240

hopefully this video has given you an insight

into how the linux password and shadow files work

00:20:24.960 --> 00:20:32.400

remember the md5 example used was only for example

purposes there are much better hash algorithms

00:20:32.400 --> 00:20:37.280

available today if you learned something new

from this video please give it a thumbs up

00:20:37.920 --> 00:20:42.720

if you've not already subscribed please do so

click the notification icon to get notified when

00:20:42.720 --> 00:20:49.760

i create my next video thanks for watching and

i look forward to seeing you on a future video

Previous Application Security Testing
Application Security Testing