Private Key Formats: Why Your Key Gets Rejected

You have a valid private key. You can see it right there in the file. But when you try to import it into your application, you get an error like PrivateKeyInfo cannot be cast to PEMKeyPair or unsupported key format. You didn't do anything wrong -- your key is just in the wrong format.

Two formats, one file extension

Private keys in PEM files come in two formats, and they both use the .pem or .key extension. The only reliable way to tell them apart is the header line.

Traditional (PKCS#1 / SEC1)

The header includes the algorithm name:

-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEA2a2rwplBQLQ...
-----END RSA PRIVATE KEY-----

For EC keys:

-----BEGIN EC PRIVATE KEY-----
MIGkAgEBBDDyz5m/zyQ29eqaz+X...
-----END EC PRIVATE KEY-----

This is the older format. Each algorithm has its own ASN.1 structure and its own header. RSA keys follow PKCS#1 (RFC 8017), EC keys follow SEC1 (RFC 5915).

PKCS#8

The header is generic -- no algorithm name:

-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQE...
-----END PRIVATE KEY-----

This looks the same whether it's RSA, EC, or Ed25519. The algorithm identifier is encoded inside the ASN.1 structure, not in the header. PKCS#8 is defined in RFC 5958.

There's also an encrypted variant:

-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIFHDBOBgkqhkiG9w0BBQ0wQTA...
-----END ENCRYPTED PRIVATE KEY-----

Why this keeps catching people

OpenSSL changed its default output format between versions 1.x and 3.x.

The genrsa change is the one that breaks things. The same command, the same arguments, but a different output format depending on which OpenSSL version is installed. If you upgraded your server or moved to a container with a newer base image, your keys silently changed format.

What accepts what

Traditional format works with

• Older Java applications (Tomcat, JBoss, Spring) using BouncyCastle PEM parsing
• PowerSchool, Blackboard, and other education platforms
• F5 BIG-IP, Citrix ADC, and older load balancers
• OpenSSL 1.x tools and scripts
• nginx and Apache (both formats work, but traditional has been the convention)
• OpenVPN
• Most hardware security modules (HSMs)

PKCS#8 format works with

• Go (standard library crypto/tls)
• .NET / C# (X509Certificate2)
• Python (cryptography library)
• AWS Certificate Manager, GCP, Azure
• Kubernetes secrets and ingress
• Modern Java (Java 11+ with updated BouncyCastle)
• OpenSSL 3.x tools
• nginx and Apache (both formats work)

The overlap

nginx, Apache, HAProxy, and most modern web servers accept both formats without complaint. The problems show up in application-level code that parses the PEM file directly -- Java's BouncyCastle library is the most common offender.

How to convert between formats

Traditional to PKCS#8

# RSA
openssl pkey -in traditional.key -out pkcs8.key

# EC
openssl pkey -in traditional.key -out pkcs8.key

The pkey command always outputs PKCS#8.

PKCS#8 to Traditional

# RSA
openssl rsa -in pkcs8.key -out traditional.key -traditional

# EC
openssl ec -in pkcs8.key -out traditional.key

Remove password from a key

# RSA (encrypted traditional)
openssl rsa -in encrypted.key -out decrypted.key

# Any format (via pkey, outputs PKCS#8)
openssl pkey -in encrypted.key -out decrypted.key

Quick check: what format is my key?

head -1 your.key

What about Ed25519?

Ed25519 keys only exist in PKCS#8 format. There is no traditional format for Ed25519 -- it was defined after PKCS#8 became the standard, so there was never a reason to create an algorithm-specific structure.

-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEIGfCOl...
-----END PRIVATE KEY-----

If your application can't handle PKCS#8, it can't use Ed25519 keys.

The getaCert approach

When you generate a certificate on getaCert.com, we give you both formats:

• Private Key (.pkey) -- Traditional format, works with the widest range of tools
• Private Key (PKCS#8) (.pkcs8.pkey) -- Modern format, for applications that prefer it

Download whichever one your application needs. If you're not sure, start with the traditional .pkey file -- it's the safer bet.