From 3a90bed728993e14b774f74c89d537b86dfa8d5f Mon Sep 17 00:00:00 2001
From: Xuan Gu <xuagu37@gmail.com>
Date: Fri, 21 Oct 2022 21:21:07 +0200
Subject: [PATCH] Update README.md

---
 README.md | 1 +
 1 file changed, 1 insertion(+)

diff --git a/README.md b/README.md
index 7bae338..4f274f9 100644
--- a/README.md
+++ b/README.md
@@ -118,3 +118,4 @@ when batch_size is large (16, 32, 64, 128), throughput_amp > throughput_tf32.
 - For multi-node benchmarking, we need to use "srun" command; also, the line "#SBATCH --ntasks-per-node=8" has to been added. Otherwise the process will hang.
 - Benchmarking with dim = 2, nodes = 1, gpus = 8, batch_size = 128 takes ~2mins.  
 If we want to finish it within a minute, we can change the number of batches from 150 (the default value) to a smaller number. Or we can try some smaller datasets.
+- On single node, max batch_size is 256; on multi-node, max batch_size is 128.
-- 
GitLab