{"id":2739,"date":"2012-09-20T08:47:38","date_gmt":"2012-09-20T07:47:38","guid":{"rendered":"http:\/\/www.frogsinspace.at\/?p=2739"},"modified":"2025-01-10T12:13:36","modified_gmt":"2025-01-10T11:13:36","slug":"research-automatic-environment-adaptive-renderings-and-composition","status":"publish","type":"post","link":"https:\/\/www.frogsinspace.at\/?p=2739","title":{"rendered":"Research: automated realworld-environment matching rendering and compositing"},"content":{"rendered":"<p id=\"top\" \/>&nbsp;<\/p>\n<p>In 2012 i got contacted by the company behind <a href=\"http:\/\/www.panomax.at\/\">Panomax<\/a>, the provider of highres 360\u00b0&nbsp; panoramic live camera streams, to do some research on how to integrate 3d rendered content in their panocam live streams. This would enable them to place virtual, nonexisting objects for any thinkable purpose inside their live panoramas.<\/p>\n<p>The goal was create renderings which respect the ever changing light and weather conditions, and still produce a plausible image composition every few minutes. Thus creating an automated solution that does all this in a fraction of the panocam update timeframe was key. So i came up with a scripted backend for 3ds Max, which solves this using&nbsp; the following steps:<\/p>\n<ul>\n<li>launch 3ds Max in headless mode and instruct it to watch a specific folder for incoming 360\u00b0 panoramic images<\/li>\n<li>pull new panoramic images ( usually well above 6K in resolution ) into a customized 3d Max scene<\/li>\n<li>Adapt the scene&#8217;s environment and lighting conditions depending on time of day, season and weather conditions<\/li>\n<li>render the region of interest (virtual object&#8217;s location), using matte techniques catching shadows and lighting<\/li>\n<li>compose the rendered output onto the original panoramic image at correct position<\/li>\n<li>send updated panoramic image to the Panomax hosting server<\/li>\n<li>loop through above steps indefinitely<\/li>\n<\/ul>\n<p>Finally here are some image sequences produced by this method:<\/p>\n<p><a href=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/\/2012\/09\/P3D_Cars_At_DifferenteTime.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-2745 aligncenter\" src=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/\/2012\/09\/P3D_Car_RealVsRendered.jpg\" alt=\"P3D_Car_RealVsRendered\" width=\"2000\" height=\"466\" srcset=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/2012\/09\/P3D_Car_RealVsRendered.jpg 2000w, https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/2012\/09\/P3D_Car_RealVsRendered-384x89.jpg 384w, https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/2012\/09\/P3D_Car_RealVsRendered-1024x239.jpg 1024w\" sizes=\"auto, (max-width: 2000px) 100vw, 2000px\" \/><\/a><br \/>\n<a href=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/\/2012\/09\/P3D_Cars_At_DifferenteTime.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-2754\" src=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/\/2012\/09\/P3D_Cars_At_DifferenteTime.jpg\" alt=\"P3D_Cars_At_DifferenteTime\" width=\"1066\" height=\"533\" srcset=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/2012\/09\/P3D_Cars_At_DifferenteTime.jpg 1066w, https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/2012\/09\/P3D_Cars_At_DifferenteTime-384x192.jpg 384w, https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/2012\/09\/P3D_Cars_At_DifferenteTime-1024x512.jpg 1024w\" sizes=\"auto, (max-width: 1066px) 100vw, 1066px\" \/><\/a><br \/>\nThe following example image shows a 3d rendered xDrive Cup column seamlessly integrated in the scenerie<\/p>\n<p><a href=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/\/2012\/09\/P3D_xDrive_Cup.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-2747 aligncenter\" src=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/\/2012\/09\/P3D_xDrive_Cup.jpg\" alt=\"P3D_xDrive_Cup\" width=\"800\" height=\"490\" srcset=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/2012\/09\/P3D_xDrive_Cup.jpg 800w, https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/2012\/09\/P3D_xDrive_Cup-384x235.jpg 384w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/a><\/p>\n<p>Here&#8217;s a full day sequence with 3d rendered architecture and a red BMW&nbsp; integrated in the environment<\/p>\n<p style=\"text-align: center;\"><a href=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/\/2012\/09\/P3D_ArchCarTest.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-2743 aligncenter\" src=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/\/2012\/09\/P3D_ArchCarTest.jpg\" alt=\"P3D_Arch&amp;CarTest\" width=\"1600\" height=\"687\" srcset=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/2012\/09\/P3D_ArchCarTest.jpg 1600w, https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/2012\/09\/P3D_ArchCarTest-384x165.jpg 384w, https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/2012\/09\/P3D_ArchCarTest-1024x440.jpg 1024w\" sizes=\"auto, (max-width: 1600px) 100vw, 1600px\" \/><\/a><\/p>\n<div style=\"width: 642px;\" class=\"wp-video\"><video class=\"wp-video-shortcode\" id=\"video-2739-1\" width=\"642\" height=\"356\" loop preload=\"auto\" controls=\"controls\"><source type=\"video\/mp4\" src=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/\/2012\/09\/P3D_ArchCarTest.mp4?_=1\" \/><a href=\"https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/\/2012\/09\/P3D_ArchCarTest.mp4\">https:\/\/www.frogsinspace.at\/wp1\/wp-content\/uploads\/\/2012\/09\/P3D_ArchCarTest.mp4<\/a><\/video><\/div>\n","protected":false},"excerpt":{"rendered":"<p>&nbsp; In 2012 i got contacted by the company behind Panomax, the provider of highres 360\u00b0&nbsp; panoramic live camera streams, to do some research on how to integrate 3d rendered content in their panocam live streams. This would enable them&hellip; <\/p>\n","protected":false},"author":1,"featured_media":2754,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[8,10,16,6],"tags":[],"class_list":["post-2739","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-work_3d","category-work_code","category-experiments","category-scriptsplugins"],"_links":{"self":[{"href":"https:\/\/www.frogsinspace.at\/index.php?rest_route=\/wp\/v2\/posts\/2739","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.frogsinspace.at\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.frogsinspace.at\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.frogsinspace.at\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.frogsinspace.at\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2739"}],"version-history":[{"count":13,"href":"https:\/\/www.frogsinspace.at\/index.php?rest_route=\/wp\/v2\/posts\/2739\/revisions"}],"predecessor-version":[{"id":3890,"href":"https:\/\/www.frogsinspace.at\/index.php?rest_route=\/wp\/v2\/posts\/2739\/revisions\/3890"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.frogsinspace.at\/index.php?rest_route=\/wp\/v2\/media\/2754"}],"wp:attachment":[{"href":"https:\/\/www.frogsinspace.at\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2739"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.frogsinspace.at\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2739"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.frogsinspace.at\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2739"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}