Recurrent Neural Networks (RNN)
Explore Recurrent Neural Networks for sequential data processing. This is a foundational concept in artificial intelligence and machine learning that professional developers rely on daily. The explanations below are written to be beginner-friendly while covering the depth and nuance that comes from real-world AI/ML experience. Take your time with each section and practice the examples
What are RNNs?
Recurrent Neural Networks (RNNs) are designed to work with sequential data. They have connections that form directed cycles, allowing them to maintain internal memory.. This is an essential concept that every AI/ML developer must understand thoroughly. In professional development environments, getting this right can mean the difference between code that works reliably and code that breaks in production. The following sections break this down into clear, digestible pieces with practical examples you can try immediately
Key Components
- Hidden State: Memory of previous inputs
- Input Gate: Controls new information
- Forget Gate: Controls what to forget
- Output Gate: Controls what to output
Implementation
import tensorflow as tf
from tensorflow import keras
import numpy as np
import matplotlib.pyplot as plt
# Generate sample sequential data
np.random.seed(42)
X = np.random.randn(1000, 50, 10) # 1000 sequences, 50 time steps, 10 features
y = np.random.randint(0, 5, 1000) # 5 classes
# Convert to one-hot encoding
y = tf.keras.utils.to_categorical(y, 5)
# Split the data
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Create RNN model
model = keras.Sequential([
keras.layers.LSTM(64, return_sequences=True, input_shape=(50, 10)),
keras.layers.LSTM(32),
keras.layers.Dense(64, activation='relu'),
keras.layers.Dropout(0.2),
keras.layers.Dense(5, activation='softmax')
])
# Compile the model
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
# Train the model
history = model.fit(X_train, y_train,
epochs=10,
batch_size=32,
validation_split=0.2,
verbose=1)
# Evaluate the model
test_loss, test_accuracy = model.evaluate(X_test, y_test, verbose=0)
print(f"Test accuracy: {test_accuracy:.4f}")