Foundry Labs / From the community
AI Infrastructure Planning: Follow the Power and Data Constraints
In this article
A compute plan can look efficient on a spreadsheet and still be impossible to build. At Tokyo AI's Beyond Silicon event, Karthik Rampalli's data-center talk placed AI infrastructure inside constraints of electricity, land, water and location. These are not background details once model demand grows; they shape which services can be delivered, where and at what cost.
Follow the physical path of a request
AI service design usually starts with model size, latency and throughput. A data center has to supply power, cooling, networking and space for that service under normal load and peaks. If the proposed capacity depends on energy or a site that cannot be secured on the deployment timeline, a benchmark result cannot rescue the plan. The infrastructure review should therefore connect application demand to hardware, facility capacity and the grid.
Location introduces another tradeoff. Placing compute near users can reduce some latency and data-transfer costs, while distributing it across regions may improve resilience or access to power. Data movement, sovereignty requirements and operational support can push in different directions. Karthik's talk treated the data center as active infrastructure, not a passive room in which servers happen to sit.
Review a workload before committing capacity
Imagine a team moving a document-analysis service from trial traffic to a steady enterprise workload. It can estimate requests per day, peak concurrency, input size and acceptable delay, then test how different model and cache choices change the compute requirement. The next review asks where the service will run, what capacity is actually available and what happens when a site loses power or connectivity. This is an illustrative planning exercise, not a cost or deployment result from the event.
For Foundry Labs and its clients, the useful question is whether the whole system is sustainable at the intended scale. Hardware acceleration matters, but so do the facility, energy contract and recovery plan behind it. The physical limits should be visible before an AI pilot is promised as a live service.
Tokyo AI is a community of builders, researchers and business leaders in Japan, run by Foundry Labs. See the event page.
Materials from the speakers’ talks are available upon request. Please use our contact form and include the event title and speaker name.
Speakers
- Karthik Rampalli, Head of Japan, Ceremorphic.