Looking for a way to pull the last update time of all mysql databases rather than querying each one individually? I had this question posed to me earlier today and decided to document this here as my google searches were returning lots of advice that was close, but not really what I thought should be a fairly straight forward question.

After a bit of fiddling around, I finally came to:


For example:

MariaDB [(none)]> select * from (SELECT UPDATE_TIME, TABLE_SCHEMA FROM information_schema.tables where UPDATE_TIME IS NOT NULL ORDER BY TABLE_SCHEMA, UPDATE_TIME desc)Q group by TABLE_SCHEMA;
| 2017-01-02 09:50:56 | information_schema |
| 2016-07-14 10:45:23 | mysql |

Or from a test environment I spun up to make sure I was getting the actual results I wanted.

mysql> select * from (SELECT UPDATE_TIME, TABLE_SCHEMA FROM information_schema.tables where UPDATE_TIME IS NOT NULL ORDER BY TABLE_SCHEMA, UPDATE_TIME desc)Q group by TABLE_SCHEMA;
| 2017-01-02 09:44:32 | db1 |
| 2017-01-02 09:50:10 | db2 |
| 2017-01-02 08:49:48 | db3 |
| 2016-09-14 14:12:11 | mysql |
| 2017-01-02 09:46:37 | db4 |
5 rows in set (0.01 sec)

Using this query, you should be able to see at a glance what the last update time for every database on your system was.

Category: mysql

If you’ve ever encountered this, you probably spent a fair amount of time scratching your head over it before it got resolved. Recently, I’ve seen this issue arise twice with clients, so I figured it was time to document it so that perhaps a google search might hit this record and help others.

If you’re currently finding that your curl requests resolve to your server rather than giving you a “Couldn’t resolve host” error, I’m betting the following details are true:

1) You have a wildcard DNS entry for your domain
2) That domain with the wildcard entry is your “search” line in /etc/resolv.conf

What this looks like:

First, we pick a domain we know doesn’t exist:

(~) # dig thisdontexistanywhereatall.com +short
(~) #

Then we curl it:

(~) # curl -I thisdontexistanywhereatall.com
HTTP/1.1 301 Moved Permanently
Date: Sun, 21 Feb 2016 19:44:33 GMT
Server: Apache/2.2.15 (CentOS)
Location: http://alex.darke.net/
Connection: close
Content-Type: text/html; charset=iso-8859-1

What? Why is a non-existent domain being responded to by MY server?

What’s happening is this:

A) The curl request looks up the domain and finds no record for it
B) the DNS libraries called then assume that it might be a subdomain and so it checks the search domain defined in /etc/resolv.cnf
C) because there is a wildcard DNS entry for my domain, it results in a match for thisdontexistanywhereatall.com.darke.net

To fix this, you can simply append a . to the end of the domain you are curling. This keeps step B and C from happening:

(~) # curl thisdontexistanywhereatall.com.
curl: (6) Couldn't resolve host 'thisdontexistanywhereatall.com.'

To prevent having to append that . each and every time, you can also elect to put a domain without a wildcard DNS entry in the /etc/resolv.conf.

Hopefully this helps the next guy.

Category: curl, dns, linux

I’m seeing more and more questions related to this and wanted to put how to fix it out there for folks.

Gmail has started flagging emails that were sent without encryption using a small red lock. It looks like this:

Screenshot from 2016-02-17 12:23:16

To fix this is pretty easy with Postfix.

  1. Log in to the server
  2. Open up /etc/postfix/main.cf
  3. Ensure the following values are set:
  4. smtpd_tls_security_level = may
    smtp_tls_security_level = may
    smtp_tls_loglevel = 1
    smtpd_tls_loglevel = 1

    (there are three levels you can set: none, may, or encrypt. Using may ensures we don’t lock out those who can’t do encryption)

  5. Reload postfix: service postfix reload
  6. Send a test email and check gmail. You should now see that there is no broken lock:

Screenshot from 2016-02-17 12:27:08

Category: encryption, mail

Sanction, my little script for on the fly blocking of an entire ip set belonging to a specific country, has been updated to include ipset support if ipset is installed.



There’s still a few items on the wish list for this. Correct chaining in iptables for servers without ipset, setting up a clear and logical way to persist the ipset rules through a reboot, etc. For now, if you are looking for a quick and easy way to ban a country from accessing a port (or all ports) on your server, with iptables or ipset, sanction may be what you are looking for.

Category: iptables

One of the more useful tools in helping to locate when/where a server was compromised are manual anti-virus scans. Additionally, a great hurdle to put up in front of malicious attackers is a scanner that scans files as they are created/modified to check for malicious content.

In the linux world, there’s not always a great understanding of what options are available, so I’m very briefly going to outline three that I am aware of and/or use.

The three I am going to cover are:

Maldetect (LMD)


Category: advice, antivirus, linux

# gdb -p (pid of mysql) -ex "set max_connections=200" -batch

Note, this is the PID of mysql, not mysql_safe.

Very handy for when you need to make that quick adjustment but don’t have permission to restart the database.

Category: advice, mysql

If you’re like me, you’re seeing more and more requests to disable plaintext authentication without encryption for POP and IMAP due to the higher number of people going for PCI compliance.

Fortunately, in Dovecot, this is very easy to configure. The setting you want to use is:

disable_plaintext_auth = yes

This can be found in /etc/dovecot/conf.d/10-auth.conf on newer Dovecot installs:

# Disable LOGIN command and all other plaintext authentications unless
# SSL/TLS is used (LOGINDISABLED capability). Note that if the remote IP
# matches the local IP (ie. you’re connecting from the same computer), the
# connection is considered secure and plaintext authentication is allowed.
#disable_plaintext_auth = yes

Restart the service after uncommenting the directive out, and you’re golden.

Edit: The conf.d/ folder seems to have appeared with Dovecot 2.x versions. If you’re using Dovecot 1.x, you should be able to plug the line into your /etc/dovecot.conf file without issue.

Category: advice, mail

Just a little useful trick, if you’ve ever had need of it.

If you have files in lsof that are “deleted”, but still being held open by a process, it is actually possible to restore that file.

To show you what I mean, I have created a basic test file:

[darke@pabu ~]$ cat file
this is my test file

In one terminal, I open that file with less. In another, once the file is open in less, I delete the file. If I search lsof for deleted files with that file name:

[darke@pabu ~]$ lsof | grep deleted | grep file
less 8351 darke 4r REG 253,3 22 118 /home/darke/file (deleted)

Now, if I close less, the file will vanish forever. However, as long as less has it open, I can pull the file out of /proc based on the above information. The relevant fields you need are the second and fourth (process ID and file descriptor):

[root@pabu ~]# cp /proc/8351/fd/4 /home/darke/file-restore
[root@pabu ~]# cat /home/darke/file-restore
this is my test file

This is a limited use case, as it depends on the file still being open by something. However, if you’ve ever been stuck trying to figure out what deleted tmp files being held open by apache are (for example), it can be quite useful.

Category: advice

First, let me preface this article with the fact that I am not a ruby developer, on rails or off. 🙂 I’m an administrator that regularly is asked to help provide the environment for Ruby on Rails developers to work in. With that caveat out of the way, let’s get to it, shall we?

This was tested on Redhat 5.10 and 6.5. By extension, that means that it should install pretty painlessly on CentOS and Fedora as well using these instructions. As for others (Ubuntu, Debian, etc), you should be able to generally follow this article and apply it to other distributions with a few changes to what you use for your package manager.

Let’s do this.


Category: ruby

If you’re seeing this post, most likely you have been googling some combination of items that reflect that you’re seeing a 60 second connection reset when downloading files directly from apache. Say, via wget:

33% [================================> ] 54,299,699 932KB/s in 61s

2014-02-09 11:08:14 (874 KB/s) - Connection closed at byte 54299699. Retrying.

Hopefully I can save you some time here. Are you running varnish in front of apache? If so, your answer is a default value for varnish’s “send_timeout” of 60 seconds. To correct it:

Modify your /etc/default/varnish in Ubuntu or /etc/sysconfig/varnish for Redhat flavors. Change it from

DAEMON_OPTS="-a :80 \
-f /etc/varnish/default.vcl \
-T \
-u varnish -g varnish \
-t 120 \
-w 5,500,300 \
-s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G \
-S /etc/varnish/secret \

to this: (Notice the added -p line)

DAEMON_OPTS="-a :80 \
-f /etc/varnish/default.vcl \
-T \
-u varnish -g varnish \
-t 120 \
-w 5,500,300 \
-s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G \
-S /etc/varnish/secret \
-p send_timeout=900 \

900 = 15 minutes. You should be able to tell right away if this was the solution to your problem and adjust the variable to the one you want to keep long term.

Please note: The above blocks of text are just examples from one of my boxes. You may have other values in there. Ultimately, the above is just provided to show you that you’re sticking one more line in the middle there. Don’t just cut and paste my entire block of text into your file, you could potentially overwrite other changes you’ve made. You’re just looking to insert that -p line with the send_timeout value.

Category: varnish


gives good tech

Kale is one of the smartest people I know

Racker Hacker
Major is always good for leet deetz