Apache SSL not working

Apache SSL not working

Apache 2.2 on debian-based Linux
# a2enmod ssl && service apache2 restart
According to Sheilds UP!, "Your computer has responded that this port exists but is currently closed to connections" for port 443.
Apache is listening on ports 80 and 443 according to netstat -nutlp
Hosting the same website on port 80 and port 443
http://website.com works, https://website.com does not
My virtual host setup is as follows. Please note that as far as I can tell the <VirtualHost *:80 *:433> line is not the problem
<VirtualHost *:443>
    SSLEngine               On
    SSLCertificateKeyFile   SSL_DIR/ssl.key
    SSLCertificateFile      SSL_DIR/ssl.crt
</VirtualHost>

<VirtualHost *:80 *:443>
    ServerAdmin     webmaster@localhost
    ServerAlias     website.com *.website.com
    ServerName      www.website.com

    # Just a few connection resets so that I don't waste my bandwidth on "hackers"
    SecRuleEngine On
    SecRule &REQUEST_HEADERS:User-Agent     "@eq 0"         drop,phase:1
    SecRule REQUEST_HEADERS:User-Agent      "^$"            drop,phase:1

    SecRule REQUEST_LINE                    "://"           drop,phase:1

    SecRule REQUEST_URI                     "^/admin"       drop,phase:1
    SecRule REQUEST_URI                     "^/mail"        drop,phase:1
    SecRule REQUEST_URI                     "^/webmail"     drop,phase:1

    DocumentRoot /path/public_html
    <Directory /path/public_html/>
            Options FollowSymLinks MultiViews
            Order allow,deny
            allow from all
    </Directory>
</VirtualHost>
I also got 2 .pem files when I registered my SSL certificate, and can't find anything about what to do with them. I'd also like to mention that this adds nothing ot the Apache error logs or syslogs.
My question is, why is SSL not working? Is it related to the .pem files?