Ubuntu 8.04 “Bug” With MySQL and AppArmor
This one drove me nuts FOR HOURS! I tried setting up my new LAMP server with the newly-released Ubuntu 8.04. When it came time to switch the configuration files so that Apache and MySQL would use /srv instead of /var/lib, Apache worked but MySQL did not.
The prolonged brain-strain stemmed from these three things:
- I insist on using custom directories.
- Debian switches logging from MySQL’s mysql.log and mysql.err files under /var/log/mysql and into /var/log/syslog.
- AppArmor is something new to Ubuntu, so I wasn’t familiar with what its logs look like.
- The logs generated by tripping AppArmor aren’t too specific.
I’ve included a copy of my logs so you can see why. I’ve color-coded things so you won’t have a headache trying to decipher what’s going on. The time stamp is blue, machine name is red, messages from MySQL are in green, and kernel messages due to AppArmor are in purple.
Apr 24 21:03:05 sara mysqld_safe[7586]: started
Apr 24 21:03:05 sara kernel: [ 1156.950776] audit(1209085385.976:14): type=1503 operation=”inode_create” requested_mask=”w::” denied_mask=”w::” name=”/srv/mysql/sara.lower-test” pid=7588 profile=”/usr/sbin/mysqld” namespace=”default”
Apr 24 21:03:05 sara mysqld[7589]: 080424 21:03:05 [Warning] Can’t create test file /srv/mysql/sara.lower-test
Apr 24 21:03:05 sara kernel: [ 1156.955085] audit(1209085385.980:15): type=1503 operation=”inode_create” requested_mask=”w::” denied_mask=”w::” name=”/srv/mysql/sara.lower-test” pid=7588 profile=”/usr/sbin/mysqld” namespace=”default”
Apr 24 21:03:05 sara mysqld[7589]: 080424 21:03:05 [Warning] Can’t create test file /srv/mysql/sara.lower-test
Apr 24 21:03:06 sara kernel: [ 1157.018500] audit(1209085386.044:16): type=1503 operation=”inode_permission” requested_mask=”rw::” denied_mask=”rw::” name=”/srv/mysql/ibdata1″ pid=7588 profile=”/usr/sbin/mysqld” namespace=”default”
Apr 24 21:03:06 sara mysqld[7589]: 080424 21:03:06 InnoDB: Operating system error number 13 in a file operation.
Apr 24 21:03:06 sara mysqld[7589]: InnoDB: The error means mysqld does not have the access rights to
Apr 24 21:03:06 sara mysqld[7589]: InnoDB: the directory.
Apr 24 21:03:06 sara mysqld[7589]: InnoDB: File name ./ibdata1
Apr 24 21:03:06 sara mysqld[7589]: InnoDB: File operation call: ‘open’.
Apr 24 21:03:06 sara mysqld[7589]: InnoDB: Cannot continue operation.
Apr 24 21:03:06 sara mysqld_safe[7597]: ended
Hindsight being 20/20, it’s easy to see what’s going on now. MySQL was trying to access something, and AppArmor was saying, “No!” To those of us new to this, it looks like the MySQL process might not have the proper access rights to the files or directories it needs to have access to. I stared for double-digit minutes comparing access rights between /var/lib/mysql and /srv/mysql. It was only after I started Googling that I came across the bug report for Ubuntu 8.04 while it was in beta.
From now on, I’m mounting my partitions to /var/lib instead of /srv. 
Jenny La, 
jeremy, 
Lost In Tokyo 8:03 am on April 28, 2008 Permalink |
Here’s a fix on Ubuntu – assuming you are trying to mount to /srv/mysql
1. sudo vi /etc/apparmor.d/usr.sbin.mysqld
Look for the two lines:
/var/lib/mysql/ r,
/var/lib/mysql/** rwk,
In my file, they are lines 27 and 28.
You want to mimick these two lines, so add:
/srv/mysql/ r,
/srv/mysql/** rwk,
on lines 29 and 30.
2. sudo /etc/init.d/apparmor restart
3. sudo /etc/init.d/mysql restart
and you should be up-and-running. Good luck!
Jenny La 2:12 pm on December 9, 2011 Permalink |
omg i want to give you a hug so bad. thank you !!!
brainwreckedtech 1:34 pm on April 28, 2008 Permalink |
I already have things working. The solution was in the bug report, but it’s kinda buried and I should have posted the solution here. That’s what I get for posting at 4:43 in the morning.
Sublivion2k 9:24 pm on May 5, 2008 Permalink |
THANK YOU THANK YOU THANK YOU!!!!
This one really sucked to solve – you are a king!
t o b e 5:49 am on May 14, 2008 Permalink |
Thanks from me also. I had a similar problem where I was trying to start a local instance of mysqld for testing purposes in a subdir of my home account. In my case I had no logs from apparmor to look at so all I knew was there was permission problem that even running as root wouldn’t solve.
Props to you, sir..
Alex Ef 2:38 pm on June 3, 2008 Permalink |
Thank you. It took also hours for me to find out it wasn’t my fault.
jeremy 1:13 pm on June 19, 2008 Permalink |
Thank you! This took me absolutely for-fucking-ever to figure out.
chompomonim 5:07 pm on July 18, 2008 Permalink |
Big thanx. You save my life
Tractor 9:55 am on August 8, 2008 Permalink |
Excellent information, well done and thanks. It helped a lot but was not the full story in my case as I am using DRBD. I have stopped the Apparmor service and this is working.
RKA KriK 4:54 pm on August 18, 2008 Permalink |
Lost In Tokyo, doooode! Big-big-big thanks!!!
Changing the data directory for MySQL in Ubuntu 6:55 am on September 23, 2008 Permalink |
[...] – Ubuntu Wiki The Brainwrecked Tech had the same problem Ubuntu bug #201799 A thread on the Ubuntu forums with the [...]
uzahnd 2:17 pm on September 25, 2008 Permalink |
This works well if the datadir is on the same partition as the root-system, but I can’t get this working on a mysql-Server located on an other partition mounted in /media/SERVER/
anyone to know why?
brainwreckedtech 4:44 pm on September 25, 2008 Permalink |
While not stated in my OP, I have /srv on a different partition from root, so I don’t think that’s the problem. The real question is, “Why are you using /media?” That directory is supposed to be reserved for removable media. If you are using media that is normally removable as something more permanent, consider moving it elsewhere.
As for solving your current situation as-is, I’m not sure. What are your logs saying?
uzahnd 4:45 am on October 2, 2008 Permalink |
oh, mounting the partition in /srv solves the problem. Thank you! (Until now, I mounted all external disks in media to keep track of them…)
Mark Rose 10:21 am on December 3, 2008 Permalink |
I just spent 6 hours trying to figure this shit out. THANKS!!
Dave Stevens 12:59 pm on March 13, 2009 Permalink |
I just spent 5 minutes figuring this out by searching google. Thanks for the post!
labsji 1:59 pm on May 9, 2009 Permalink |
Thanks a lot. I was addressing this problem in many different ways and finally hit upon this fix by serendipity.( via http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1663)
Thanks again.
-Balaji S.
MySQL: Ubuntu Server 9.04, moving the data directory to a different location « mundane idiosyncrasies 7:53 pm on May 11, 2009 Permalink |
[...] A little googling led me to this blog [neodon.blogspot.com] and this blog [brainwreckedtech.wordpress.com] [...]
Thanker 9:24 am on May 20, 2009 Permalink |
Thanks a lot, finally helped me in moving my mysql directory using symlinks to another location on Ubuntu
Q 10:00 am on August 3, 2009 Permalink |
Thanks for this, life saver.
Bilal Khan 2:25 am on August 5, 2009 Permalink |
THANK YOUUUUUUUUUUUUUUUUUUUUUUUUUUU!!!!!!!!!!!!!!!!!!!! IT GOT ME NUTSSSSSSS :S
Andrew S 11:16 pm on August 7, 2009 Permalink |
I’ve spent 2 hours trying to move my mysql db from /var/lib/mysql to another location on ubuntu 9.04. chown/chgrp wasn’t enough. the apparmor solution did it. thank you very much for sharing your solution!
Mike P 1:41 pm on November 19, 2009 Permalink |
I’m going to keep this going – 1 year and 7 months later – this saved me as well – Ubuntu 9.10, MySQL 5.0 that was running AppArmor. I would’ve never found the problem! The AppArmor logs were in a different file. Too bad I didn’t find this post 2 days ago! #($*#(@#*@@!
Thank you.
Leon Waldman 7:29 am on March 16, 2010 Permalink |
Thanks for that man!! :^D
Buck Golemon 11:07 pm on April 10, 2010 Permalink |
For the record, these are the apparmor lines I had to add to ‘/etc/apparmor.d/usr.sbin.mysqld’, just below the ‘/etc/mysql/my.cnf’ line:
owner /opt/SQLsandbox/data/ r,
owner /opt/SQLsandbox/data/** rwk,
owner /opt/SQLsandbox/my.cnf r,
A new server « GreenPhotons 5:26 pm on April 14, 2010 Permalink |
[...] I found this hint through Google in another blog: http://brainwreckedtech.wordpress.com. [...]
Amrac 10:27 pm on November 6, 2010 Permalink |
Thanks
Akash 1:35 am on April 27, 2011 Permalink |
Thank you so much! I had been trying different things for the last 6 hours and although I did see apparmor mentioned on some other blog, I didn’t make much of it. Because of your decision to blog about this, I have finally crossed this roadblock. A big WOOT for your blog! :^)
Ingmar 11:08 am on May 10, 2011 Permalink |
Thank you very much!