Today, NVIDIA has a total of 4 new NIM microservices with some of them tailored and dedicated to Japan and Taiwan.
The rise in AI demands more specific and localized models thus it is answered through the Llama-3-Swallow-70B training with Japanese data while Llama-3-Taiwan-70B was based on Mandarin.
Both regionalized LLMs are capable of providing a deeper understanding of local culture, laws, regulations, and customs.
On the other hand, the RakutenAI 7B model built from the foundations of Mistral-7B and trained on English and Japanese datasets, is available as two NIM microservices for Chat and Instruct. Its capability was showcased in the form of the ‘Highest Ranking in LM Evaluation Harness” benchmark dated from January to March 2024.
The achievement highlights the importance and advantages of localized LLM in providing more accurate and nuanced communication through the study and output of cultural and linguistic subtleties.
Diving into the details and it shows that these microservices are optimized for inference using the NVIDIA TensorRT-LLM open-source library and are available for access as part of the NVIDIA AI Enterprise platform in addition to the classic API way.
NIM microservices for Llama 3 70B — which serves as the base model for the new Llama–3-Swallow-70B and Llama-3-Taiwan-70B NIM microservices — offer up to 5x higher throughput. This reduces the total cost of running the models in production and improves user experiences by decreasing latency.
Not just work but also social values
Prior to the announcement of the Japan and Taiwan specialized model, they are already being deployed in production to test out markets and gather true data in the real world.
Preferred Networks from Japan is using a highly-tuned healthcare-specific version known as Llama3-Preferred-MedSwallow-70B trained with Japanese medical data and has achieved top scores on the Japan National Examination for Physicians. On the other hand, Chang Gung Memorial Hospital (CGMH) in Taiwan is creating a custom AI Inference Service (AIIS) to centralize all LLM applications within the hospital system through the power of Llama 3-Taiwan 70B that aims to enable nuanced medical language that patients can understand.
Other than the medical field, we also have Pegatron also from Taiwan currently planning to integrate Llama 3-Taiwan 70B for both internal and external-facing applications with its existing PEGAAi Agentic AI System for automation already utilizing it.
Notable examples also include global petrochemical manufacturer Chang Chun Group, world-leading printed circuit board company Unimicron, technology-focused media company TechOrange, online contract service company LegalSign.ai, and generative AI startup APMIC all being part of the Llama-3-Taiwan 70B supporter.
NVIDIA AI Foundry – One-stop AI Hub for businesses
While regional AI models can offer culturally nuanced and localized responses, enterprises often need to fine-tune them for specific business processes and domain expertise.
NVIDIA AI Foundry is a platform and service that includes popular foundation models, NVIDIA NeMo for fine-tuning, and dedicated capacity on NVIDIA DGX Cloud, providing developers with a full-stack solution for creating customized foundation models packaged as NIM microservices.
Additionally, developers using NVIDIA AI Foundry can access the NVIDIA AI Enterprise software platform, which offers security, stability, and support for production deployments.
NVIDIA AI Foundry equips developers with the necessary tools to more quickly and easily build and deploy custom, regional language NIM microservices, powering AI applications that deliver culturally and linguistically appropriate results for their users.
