C++学习:六个月从基础到就业------内存管理:自定义内存管理(上篇)
本文是我C++学习之旅系列的第二十一篇技术文章,也是第二阶段"C++进阶特性"的第六篇,主要介绍C++中的自定义内存管理技术(上篇)。查看完整系列目录了解更多内容。
引言
在前面的文章中,我们已经探讨了C++标准提供的内存管理工具,包括堆与栈的使用、new
/delete
操作符、内存泄漏的避免、RAII原则以及智能指针。这些机制在大多数应用场景下已经足够使用,但在某些特定情况下,标准内存分配器可能无法满足我们的需求,例如:
- 需要极高的性能,标准分配器造成的开销过大
- 需要特定的内存布局或对齐方式
- 需要减少内存碎片化
- 需要追踪和调试内存使用
- 在特定硬件或嵌入式系统上工作
在这些情况下,自定义内存管理就变得非常重要。本文(上篇)将详细介绍C++中自定义内存管理的基础知识、核心技术以及简单实现,而下篇将会介绍更多高级应用场景和实际项目中的最佳实践。
内存管理基础回顾
在深入自定义内存管理之前,让我们简要回顾一下C++中内存管理的基础知识。
标准内存分配过程
当我们在C++中使用new
运算符时,实际发生了以下步骤:
- 内存分配 :调用
operator new
函数分配原始内存 - 对象构造:在分配的内存上调用构造函数
- 返回指针:返回指向新创建对象的指针
类似地,当使用delete
运算符时:
- 对象析构:调用对象的析构函数
- 内存释放 :调用
operator delete
函数释放内存
这个过程的关键在于operator new
和operator delete
函数,它们是可以被重载的,这为我们提供了自定义内存管理的入口。
堆内存分配的问题
标准堆内存分配存在一些常见问题:
- 性能开销:每次分配/释放都需要系统调用,开销较大
- 内存碎片化:频繁的分配和释放可能导致内存碎片
- 缓存不友好:随机分配的内存可能分散在不同的缓存行中
- 分配失败处理:需要额外的代码处理内存分配失败
- 调试困难:难以追踪内存泄漏和内存使用模式
自定义内存管理的目标就是解决或缓解这些问题。
自定义内存管理技术
重载全局new和delete运算符
最直接的自定义内存管理方式是重载全局new
和delete
运算符:
cpp
#include <iostream>
#include <cstdlib>
// 重载全局operator new
void* operator new(std::size_t size) {
std::cout << "Global operator new called, size = " << size << " bytes" << std::endl;
void* ptr = std::malloc(size);
if (!ptr) throw std::bad_alloc();
return ptr;
}
// 重载全局operator delete
void operator delete(void* ptr) noexcept {
std::cout << "Global operator delete called" << std::endl;
std::free(ptr);
}
// 重载数组版本
void* operator new[](std::size_t size) {
std::cout << "Global operator new[] called, size = " << size << " bytes" << std::endl;
void* ptr = std::malloc(size);
if (!ptr) throw std::bad_alloc();
return ptr;
}
void operator delete[](void* ptr) noexcept {
std::cout << "Global operator delete[] called" << std::endl;
std::free(ptr);
}
int main() {
// 使用重载的new/delete
int* p1 = new int(42);
std::cout << "Value: " << *p1 << std::endl;
delete p1;
// 使用重载的new[]/delete[]
int* arr = new int[10];
arr[0] = 10;
std::cout << "Array first element: " << arr[0] << std::endl;
delete[] arr;
return 0;
}
输出结果类似于:
Global operator new called, size = 4 bytes
Value: 42
Global operator delete called
Global operator new[] called, size = 40 bytes
Array first element: 10
Global operator delete[] called
这种方法会影响程序中所有的内存分配,通常用于全局内存跟踪或调试。但要注意,这是一种侵入性很强的方法,可能会影响第三方库的行为,所以在生产环境中需要谨慎使用。
类特定的new和delete重载
对于特定的类,我们可以只重载该类的new
和delete
运算符,这样就不会影响其他部分的内存分配:
cpp
class MyClass {
private:
int data;
public:
MyClass(int d) : data(d) {
std::cout << "MyClass constructor called with data = " << data << std::endl;
}
~MyClass() {
std::cout << "MyClass destructor called with data = " << data << std::endl;
}
// 类特定的operator new
void* operator new(std::size_t size) {
std::cout << "MyClass::operator new called, size = " << size << " bytes" << std::endl;
void* ptr = std::malloc(size);
if (!ptr) throw std::bad_alloc();
return ptr;
}
// 类特定的operator delete
void operator delete(void* ptr) noexcept {
std::cout << "MyClass::operator delete called" << std::endl;
std::free(ptr);
}
// 还可以重载数组版本
void* operator new[](std::size_t size) {
std::cout << "MyClass::operator new[] called, size = " << size << " bytes" << std::endl;
void* ptr = std::malloc(size);
if (!ptr) throw std::bad_alloc();
return ptr;
}
void operator delete[](void* ptr) noexcept {
std::cout << "MyClass::operator delete[] called" << std::endl;
std::free(ptr);
}
// 可以添加额外参数版本,如placement new
void* operator new(std::size_t size, const char* file, int line) {
std::cout << "MyClass::operator new called from " << file << ":" << line << std::endl;
void* ptr = std::malloc(size);
if (!ptr) throw std::bad_alloc();
return ptr;
}
// 对应的delete版本
void operator delete(void* ptr, const char* file, int line) noexcept {
std::cout << "MyClass::operator delete called from " << file << ":" << line << std::endl;
std::free(ptr);
}
};
// 使用宏简化调用
#define DEBUG_NEW new(__FILE__, __LINE__)
int main() {
// 使用类特定的new/delete
MyClass* obj1 = new MyClass(42);
delete obj1;
// 使用数组版本
MyClass* arr = new MyClass[3]{1, 2, 3};
delete[] arr;
// 使用带额外参数的版本
MyClass* obj2 = DEBUG_NEW MyClass(100);
delete obj2;
return 0;
}
这种方法只影响特定类的内存分配,适用于有特殊内存需求的类,如频繁创建销毁的小对象,或者需要进行内存使用追踪的类。
placement new
Placement new是C++的一个特殊形式的new
运算符,它允许我们在预先分配的内存上构造对象:
cpp
#include <iostream>
#include <new> // 为placement new引入必要的头文件
class Complex {
private:
double real;
double imag;
public:
Complex(double r = 0, double i = 0) : real(r), imag(i) {
std::cout << "Constructor called: " << real << " + " << imag << "i" << std::endl;
}
~Complex() {
std::cout << "Destructor called: " << real << " + " << imag << "i" << std::endl;
}
void display() const {
std::cout << "Complex number: " << real << " + " << imag << "i" << std::endl;
}
};
int main() {
// 分配一块足够大的内存,不调用构造函数
char memory[sizeof(Complex)];
// 使用placement new在预分配内存上构造对象
Complex* ptr = new(memory) Complex(3.0, 4.0);
// 使用对象
ptr->display();
// 显式调用析构函数(不需要delete,因为内存不是通过new分配的)
ptr->~Complex();
// 演示数组的placement new
constexpr int arraySize = 3;
char arrayMemory[sizeof(Complex) * arraySize];
Complex* arrayPtr = reinterpret_cast<Complex*>(arrayMemory);
// 在预分配内存上构造多个对象
for (int i = 0; i < arraySize; ++i) {
new(&arrayPtr[i]) Complex(i * 1.0, i * 2.0);
}
// 使用数组中的对象
for (int i = 0; i < arraySize; ++i) {
arrayPtr[i].display();
}
// 显式调用每个对象的析构函数
for (int i = arraySize - 1; i >= 0; --i) {
arrayPtr[i].~Complex();
}
return 0;
}
Placement new的主要用途:
- 内存池实现:在预先分配的内存池中构造对象
- 嵌入式系统:在特定内存位置构造对象
- 内存映射文件:在内存映射区域构造对象
- 性能优化:避免额外的内存分配开销
- 自定义内存对齐:在特定对齐的内存上构造对象
需要注意的是,使用placement new时,我们需要手动调用析构函数,并且不使用delete来释放内存(因为内存不是通过new分配的)。
内存池技术
内存池是一种常用的自定义内存管理技术,它通过预先分配大块内存,然后按需分配小块,减少系统调用次数,提高性能。
固定大小对象的内存池
下面是一个用于管理固定大小对象的简单内存池实现:
cpp
#include <iostream>
#include <vector>
#include <cassert>
template <typename T, size_t BlockSize = 4096>
class FixedSizeAllocator {
private:
// 内存块结构
struct Block {
char data[BlockSize];
Block* next;
};
// 空闲链表的节点结构
union Chunk {
T object;
Chunk* next;
};
Block* currentBlock;
Chunk* freeList;
size_t chunksPerBlock;
size_t allocatedChunks;
public:
FixedSizeAllocator() : currentBlock(nullptr), freeList(nullptr), chunksPerBlock(0), allocatedChunks(0) {
// 计算每个块中可以容纳的对象数量
chunksPerBlock = BlockSize / sizeof(Chunk);
assert(chunksPerBlock > 0 && "Block size too small");
}
~FixedSizeAllocator() {
// 释放所有内存块
while (currentBlock) {
Block* next = currentBlock->next;
delete currentBlock;
currentBlock = next;
}
}
// 分配内存
T* allocate() {
// 如果没有空闲块,分配新的内存块
if (!freeList) {
// 分配新块
Block* newBlock = new Block;
newBlock->next = currentBlock;
currentBlock = newBlock;
// 将新块分成多个Chunk并添加到空闲列表
Chunk* chunk = reinterpret_cast<Chunk*>(currentBlock->data);
freeList = chunk;
// 构建空闲链表
for (size_t i = 0; i < chunksPerBlock - 1; ++i) {
chunk->next = chunk + 1;
chunk = chunk->next;
}
chunk->next = nullptr;
}
// 从空闲列表中获取一个Chunk
Chunk* allocatedChunk = freeList;
freeList = freeList->next;
allocatedChunks++;
// 返回指向对象空间的指针
return reinterpret_cast<T*>(allocatedChunk);
}
// 释放内存
void deallocate(T* ptr) {
if (!ptr) return;
// 将指针转换为Chunk并添加到空闲列表的头部
Chunk* chunk = reinterpret_cast<Chunk*>(ptr);
chunk->next = freeList;
freeList = chunk;
allocatedChunks--;
}
// 统计信息
size_t getAllocatedChunks() const {
return allocatedChunks;
}
};
// 使用内存池的类
class PooledObject {
private:
int data;
static FixedSizeAllocator<PooledObject> allocator;
public:
PooledObject(int d = 0) : data(d) {
std::cout << "PooledObject constructor: " << data << std::endl;
}
~PooledObject() {
std::cout << "PooledObject destructor: " << data << std::endl;
}
// 重载new和delete使用内存池
void* operator new(std::size_t size) {
assert(size == sizeof(PooledObject));
return allocator.allocate();
}
void operator delete(void* ptr) {
if (ptr) {
allocator.deallocate(static_cast<PooledObject*>(ptr));
}
}
// 显示分配统计
static size_t getAllocatedCount() {
return allocator.getAllocatedChunks();
}
void setData(int d) {
data = d;
}
int getData() const {
return data;
}
};
// 静态成员初始化
FixedSizeAllocator<PooledObject> PooledObject::allocator;
int main() {
std::cout << "Initial allocated count: " << PooledObject::getAllocatedCount() << std::endl;
// 创建一些对象
std::vector<PooledObject*> objects;
for (int i = 0; i < 10; ++i) {
objects.push_back(new PooledObject(i));
}
std::cout << "After creation, allocated count: " << PooledObject::getAllocatedCount() << std::endl;
// 删除一些对象
for (int i = 0; i < 5; ++i) {
delete objects[i];
objects[i] = nullptr;
}
std::cout << "After partial deletion, allocated count: " << PooledObject::getAllocatedCount() << std::endl;
// 创建更多对象
for (int i = 0; i < 3; ++i) {
objects.push_back(new PooledObject(i + 100));
}
std::cout << "After more creation, allocated count: " << PooledObject::getAllocatedCount() << std::endl;
// 清理剩余对象
for (auto obj : objects) {
delete obj;
}
std::cout << "After all deletions, allocated count: " << PooledObject::getAllocatedCount() << std::endl;
return 0;
}
这是一个简单的固定大小对象内存池实现,适用于大量创建和销毁相同大小对象的场景,如游戏中的粒子系统、图形渲染中的顶点等。通过内存池,我们可以显著减少内存分配的系统调用次数,从而提高性能。
简单通用内存池
下面是一个简单的通用内存池实现,可以用于分配不同大小的内存块:
cpp
#include <iostream>
#include <vector>
#include <map>
#include <cstddef>
#include <cassert>
class SimpleMemoryPool {
private:
struct MemoryBlock {
char* memory;
size_t size;
size_t used;
MemoryBlock(size_t blockSize) : size(blockSize), used(0) {
memory = new char[blockSize];
}
~MemoryBlock() {
delete[] memory;
}
// 尝试分配内存
void* allocate(size_t bytes, size_t alignment) {
// 计算对齐调整
size_t adjustment = alignment - (reinterpret_cast<uintptr_t>(memory + used) & (alignment - 1));
if (adjustment == alignment) adjustment = 0;
// 检查是否有足够空间
if (used + adjustment + bytes > size) {
return nullptr; // 块中没有足够空间
}
// 分配内存
void* result = memory + used + adjustment;
used += bytes + adjustment;
return result;
}
};
std::vector<MemoryBlock*> blocks;
std::map<void*, MemoryBlock*> allocations; // 跟踪指针到块的映射
size_t blockSize; // 默认块大小
size_t totalAllocated;
size_t totalUsed;
public:
SimpleMemoryPool(size_t defaultBlockSize = 4096)
: blockSize(defaultBlockSize), totalAllocated(0), totalUsed(0) {
// 初始化第一个块
addBlock(blockSize);
}
~SimpleMemoryPool() {
// 释放所有块
for (auto block : blocks) {
delete block;
}
blocks.clear();
allocations.clear();
}
// 分配内存
void* allocate(size_t bytes, size_t alignment = 8) {
if (bytes == 0) return nullptr;
// 对齐大小
bytes = (bytes + alignment - 1) & ~(alignment - 1);
// 尝试在现有块中分配
for (auto block : blocks) {
void* memory = block->allocate(bytes, alignment);
if (memory) {
allocations[memory] = block;
totalUsed += bytes;
return memory;
}
}
// 无可用空间,创建新块
size_t newBlockSize = std::max(blockSize, bytes * 2);
MemoryBlock* newBlock = addBlock(newBlockSize);
// 尝试在新块中分配
void* memory = newBlock->allocate(bytes, alignment);
if (memory) {
allocations[memory] = newBlock;
totalUsed += bytes;
return memory;
}
// 分配失败
return nullptr;
}
// 释放内存(实际上在这个简单实现中不会释放,只是记录)
void deallocate(void* ptr) {
if (!ptr) return;
allocations.erase(ptr);
}
// 打印池状态
void printStatus() const {
std::cout << "Memory Pool Status:" << std::endl;
std::cout << "Number of blocks: " << blocks.size() << std::endl;
std::cout << "Total allocated: " << totalAllocated << " bytes" << std::endl;
std::cout << "Total used: " << totalUsed << " bytes" << std::endl;
std::cout << "Utilization: " << (totalAllocated > 0 ? (double)totalUsed / totalAllocated * 100.0 : 0)
<< "%" << std::endl;
}
private:
// 添加新块
MemoryBlock* addBlock(size_t size) {
MemoryBlock* block = new MemoryBlock(size);
blocks.push_back(block);
totalAllocated += size;
return block;
}
};
void simpleMemoryPoolDemo() {
SimpleMemoryPool pool;
// 分配不同大小的内存
void* p1 = pool.allocate(128);
void* p2 = pool.allocate(256);
void* p3 = pool.allocate(512);
void* p4 = pool.allocate(1024);
// 检查指针
std::cout << "Pointers:" << std::endl;
std::cout << "p1: " << p1 << std::endl;
std::cout << "p2: " << p2 << std::endl;
std::cout << "p3: " << p3 << std::endl;
std::cout << "p4: " << p4 << std::endl;
// 打印池状态
pool.printStatus();
// 写入一些数据
char* cp1 = static_cast<char*>(p1);
for (int i = 0; i < 128; ++i) {
cp1[i] = i % 256;
}
// 读取数据
std::cout << "First few bytes of p1: ";
for (int i = 0; i < 10; ++i) {
std::cout << static_cast<int>(cp1[i]) << " ";
}
std::cout << std::endl;
// 释放一些内存(实际上只是记录,不会真的释放)
pool.deallocate(p2);
pool.deallocate(p4);
// 再次分配
void* p5 = pool.allocate(200);
std::cout << "p5: " << p5 << std::endl;
pool.printStatus();
}
int main() {
simpleMemoryPoolDemo();
return 0;
}
这个简单的内存池实现了一种"只增不减"的策略,即不会释放块中未使用的内存,这样的设计在某些场景下是合理的,尤其是对于内存使用模式比较固定或短暂的应用程序。
内存分析与调试工具
除了自定义内存管理,我们还可以构建内存分析与调试工具,帮助我们追踪内存使用、发现内存泄漏等问题。下面是一个简单的内存追踪器:
cpp
#include <iostream>
#include <unordered_map>
#include <string>
#include <mutex>
#include <cstdlib>
#include <vector>
// 简单的内存分配跟踪器
class MemoryTracker {
private:
struct AllocationInfo {
void* address;
size_t size;
const char* file;
int line;
bool isArray;
AllocationInfo(void* addr, size_t s, const char* f, int l, bool arr)
: address(addr), size(s), file(f), line(l), isArray(arr) {}
};
std::unordered_map<void*, AllocationInfo> allocations;
std::mutex mutex;
size_t totalAllocated;
size_t currentAllocated;
size_t peakAllocated;
size_t allocationCount;
// 单例实例
static MemoryTracker& getInstance() {
static MemoryTracker instance;
return instance;
}
// 私有构造函数(单例模式)
MemoryTracker() : totalAllocated(0), currentAllocated(0), peakAllocated(0), allocationCount(0) {}
public:
// 记录分配
static void recordAllocation(void* address, size_t size, const char* file, int line, bool isArray) {
std::lock_guard<std::mutex> lock(getInstance().mutex);
getInstance().allocations.emplace(
address,
AllocationInfo(address, size, file, line, isArray)
);
getInstance().totalAllocated += size;
getInstance().currentAllocated += size;
getInstance().allocationCount++;
if (getInstance().currentAllocated > getInstance().peakAllocated) {
getInstance().peakAllocated = getInstance().currentAllocated;
}
}
// 记录释放
static void recordDeallocation(void* address, bool isArray) {
std::lock_guard<std::mutex> lock(getInstance().mutex);
auto it = getInstance().allocations.find(address);
if (it != getInstance().allocations.end()) {
// 检查数组/非数组匹配
if (it->second.isArray != isArray) {
std::cerr << "Error: Mismatch between new/delete and new[]/delete[]!" << std::endl;
std::cerr << "Allocation: " << (it->second.isArray ? "array" : "non-array")
<< " at " << it->second.file << ":" << it->second.line << std::endl;
std::cerr << "Deallocation: " << (isArray ? "array" : "non-array") << std::endl;
}
getInstance().currentAllocated -= it->second.size;
getInstance().allocations.erase(it);
} else {
std::cerr << "Error: Attempting to free unallocated memory at " << address << std::endl;
}
}
// 打印内存使用统计
static void printStatistics() {
std::lock_guard<std::mutex> lock(getInstance().mutex);
std::cout << "\n=== Memory Usage Statistics ===" << std::endl;
std::cout << "Total Allocated: " << getInstance().totalAllocated << " bytes" << std::endl;
std::cout << "Current Allocated: " << getInstance().currentAllocated << " bytes" << std::endl;
std::cout << "Peak Allocated: " << getInstance().peakAllocated << " bytes" << std::endl;
std::cout << "Allocation Count: " << getInstance().allocationCount << " bytes" << std::endl;
std::cout << "Currently Active Allocations: " << getInstance().allocations.size() << std::endl;
}
// 打印当前内存泄漏
static void reportLeaks() {
std::lock_guard<std::mutex> lock(getInstance().mutex);
if (getInstance().allocations.empty()) {
std::cout << "\nNo memory leaks detected." << std::endl;
return;
}
std::cout << "\n=== Memory Leaks Detected ===" << std::endl;
std::cout << "Number of leaks: " << getInstance().allocations.size() << std::endl;
std::cout << "Total leaked memory: " << getInstance().currentAllocated << " bytes" << std::endl;
// 按文件和行号组织泄漏
std::unordered_map<std::string, std::vector<AllocationInfo>> leaksByLocation;
for (const auto& pair : getInstance().allocations) {
const AllocationInfo& info = pair.second;
std::string location = std::string(info.file) + ":" + std::to_string(info.line);
leaksByLocation[location].push_back(info);
}
// 打印泄漏信息
for (const auto& pair : leaksByLocation) {
const std::string& location = pair.first;
const std::vector<AllocationInfo>& leaks = pair.second;
size_t totalSize = 0;
for (const auto& leak : leaks) {
totalSize += leak.size;
}
std::cout << "\nLocation: " << location << std::endl;
std::cout << "Leaks: " << leaks.size() << ", Total size: " << totalSize << " bytes" << std::endl;
// 打印前几个泄漏的详细信息
const size_t maxDetailsCount = 5;
for (size_t i = 0; i < std::min(leaks.size(), maxDetailsCount); ++i) {
const auto& leak = leaks[i];
std::cout << " - Address: " << leak.address
<< ", Size: " << leak.size << " bytes"
<< ", Type: " << (leak.isArray ? "array" : "non-array") << std::endl;
}
if (leaks.size() > maxDetailsCount) {
std::cout << " ... and " << (leaks.size() - maxDetailsCount) << " more" << std::endl;
}
}
}
};
// 重载全局new/delete运算符
void* operator new(std::size_t size, const char* file, int line) {
void* ptr = std::malloc(size);
if (!ptr) throw std::bad_alloc();
MemoryTracker::recordAllocation(ptr, size, file, line, false);
return ptr;
}
void* operator new[](std::size_t size, const char* file, int line) {
void* ptr = std::malloc(size);
if (!ptr) throw std::bad_alloc();
MemoryTracker::recordAllocation(ptr, size, file, line, true);
return ptr;
}
// 标准版本委托给上面的版本
void* operator new(std::size_t size) {
return operator new(size, "Unknown", 0);
}
void* operator new[](std::size_t size) {
return operator new[](size, "Unknown", 0);
}
void operator delete(void* ptr) noexcept {
if (ptr) {
MemoryTracker::recordDeallocation(ptr, false);
std::free(ptr);
}
}
void operator delete[](void* ptr) noexcept {
if (ptr) {
MemoryTracker::recordDeallocation(ptr, true);
std::free(ptr);
}
}
// 带位置参数的删除版本
void operator delete(void* ptr, const char*, int) noexcept {
operator delete(ptr);
}
void operator delete[](void* ptr, const char*, int) noexcept {
operator delete[](ptr);
}
// 宏定义简化使用
#define DEBUG_NEW new(__FILE__, __LINE__)
#define new DEBUG_NEW
class AutoCleanup {
public:
~AutoCleanup() {
MemoryTracker::printStatistics();
MemoryTracker::reportLeaks();
}
};
void memoryLeakTest() {
// 分配一些内存
int* p1 = new int(42);
int* p2 = new int[10];
char* p3 = new char[100];
// 模拟内存泄漏 - 不释放p2
delete p1;
delete[] p3;
// 故意错误使用delete
// int* p4 = new int[5];
// delete p4; // 应该使用delete[]
}
int main() {
AutoCleanup cleanup; // 程序结束时自动报告泄漏
memoryLeakTest();
return 0;
}
这个内存跟踪器可以帮助我们找出内存泄漏和不正确的内存操作,是一个功能强大的调试工具。使用宏将new
替换为带有文件名和行号的版本,可以精确定位内存泄漏的位置。
小结
在本文(上篇)中,我们详细介绍了C++中自定义内存管理的基础知识、核心技术以及简单实现。我们讨论了:
- 重载全局和类特定的
new
/delete
运算符 - 使用placement new在预分配内存上构造对象
- 实现固定大小对象的内存池
- 构建简单的通用内存池
- 创建内存分析和调试工具
这些技术为我们提供了更灵活、高效的内存管理方式,特别是在性能关键的应用程序中。
在下篇文章中,我们将继续探讨更高级的内存管理技术,包括:
- 更复杂的内存池实现
- 针对STL容器的自定义分配器
- 自定义内存管理在游戏开发、嵌入式系统和高性能计算中的应用
- 多线程环境下的内存管理
- 自定义内存管理的最佳实践和性能分析
这是我C++学习之旅系列的第二十一篇技术文章。查看完整系列目录了解更多内容。